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Abstract 


Undersea technology is on the verge of equipping remotely operated vehicle (ROV) pilots 
with a three-dimensional (3-D), real-time display incorporating data from a wide variety of 
sensors including sonar (sound navigation and ranging), cameras, and lasers. Effective 
collection, computation, and presentation of this data to the pilot in a single display presents 
hardware, software and human factors problems. This thesis focuses on human factors 
issues associated with the display of information which could enhance the pilot's efficiency 
of performance. Background information on human factors engineering, 3-D computer 
graphics displays, and application of the 3-D perspective display precede the details of the 
experiment. 


Five specific display enhancements tested include altering the displayed field of view, 
providing a screen grid, displaying the current range to the target of interest, using a 
vertical color scheme, and controlling the display update rate. Seven tests measure the 
effects of these display enhancements on the simulated piloting of an ROV. The effects of 
the ROV simulation and operator learning curves are removed to compare performance 
changes due to the various enhancements directly. Operator comments during and after 
testing as well as test monitor/author observations provide insight into the experiment. Test 
result implications for system design trade-offs are discussed in detail. Recommendations 
for future research and the proposed construction of a fully equipped ROV simulator 
complete the work. 
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Chapter 1 


Introduction 


1.1 Background 


Sonar (Sound Navigation and Ranging) is a technique for underwater range finding 
that has existed for more than fifty years. The author's thirteen years of experience with 
military and commercial sonars indicate that the techniques employed to present sonar 
information to the user significantly lag the technology used in areas such as hardware 
design, beam forming, and data О, and unnecessarily limit the bandwidth of 
information transfer. The wide variety of display designs and information formats proves 
confusing to all but the most adept and experienced users. Additionally, the typical 
И соп of the presentation to two coding dimensions, coincident with the physical 
dimensions of the cathode ray tube (CRT), severely restricts the ability to effectively 
convey large amounts of data. 

The Woods Hole Oceanographic Institution's (WHOI) Deep Submergence Labora- 
tory (DSL) operates several seagoing craft including remotely operated vehicles (ROVs), 
whose pilot "flies" the deep-diving ROV from an operations center located on a mother ship 
on the surface above. Within video camera range the pilot's display resembles the view a 
driver sees out the front windshield of an automobile, with some loss of depth perception. 
This video display format is "natural" for the pilot and provides a large information transfer 
bandwidth. Suspended sediment near the bottom frequently limits the already restricted 


visual range of camera equipment to a couple of meters. Beyond visual range the pilot 
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must rely on the customary two-dimensional (2-D) active (transmit and receive) sonar 
presentations. 

Technological advances currently provide the capability to combine undersea data 
from a number of sensors including active sonars, video tape and electronic still cameras, 
manipulators, and laser range finders. These data can be assimilated and processed in real 
time to provide large quantities of high quality information. The data generated can be used 
to provide the ROV pilot with a real-time, three-dimensional (3-D), perspective display 
similar to that of the video camera when beyond visual range. This more natural view 
would ease the pilots workload and improve the efficiency of work conducted beyond 
visual camera range. Such a perspective-view display is expected to be dominated in the 
foreground by high-resolution video data, gradually transitioning to dominance by other 
sensors as the limits of video range are encountered. The envisioned 3-D display must be 


carefully designed to best improve the ROV pilot's efficiency. 


1.2 Research Objectives 


A variety of display enhancements have been used in military and commercial 
sonars. This thesis looks at the individual effects of six specific display enhancements: 
widening the field of view (FOV) to provide more information to the pilot, 2-D and 3-D 
gridding for improved angular and depth perception, vertical color schemes for better 
altitude versus depth perception, direct range readout to shorten the search and location 
cycle, and shortening the display update rate. Seven experiments to isolate and quantify the 
individual effects of these enhancements are described in detail. The analysis allows direct 


comparison of the effectiveness of the six enhancements. 
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1.3 


Terminology 


Terminology is critical to comprehension. Several important definitions include: 
e altitude - the distance above the floor of the ocean. 


e data - "raw" numbers from which information can be extracted. 


e data point - one of a set of 25 x, y, and z coordinates chosen for a target course. 


e depth - the perceived distance into a video or computer display. This is a spatial 
perception of range. 

e display - the computer screen dedicated to graphics information. 

e driving - the act of operating the simulated ROV for testing. 

e heading - the compass direction in which the vehicle is pointed. 

e monitor - the computer screen dedicated to printed information, including update 
indication, "target hit" and "mission complete" indication, target range 
indication @ 6), and тоо world-coordinate position, heading, and 
velocities. This term is also used to describe the actions of the author as 
the testing supervisor. 

e Operator - a test subject who drove the simulated ROV. 

e pilot - a qualified flyer of operational ROVs. 

e piloting - the act of flying a operational ROV. 

e target - the numbered, cubic wireframe, data point visualizing tool. 

e target course - one of 19 (lettered A-R, X) specific sequences of data points and 
associated targets. 

e update - the complete refreshing of a computer screen. 

e vertical - the direction opposite to that in which gravity acts. A vertical color 


scheme encodes altitude information using a color scale or look-up table. 
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1.4 Overview 


Chapter 2 provides background information on human factors engineering and its 
relationship to the thesis work. 

Chapter 3 contains information on three-dimensional computer graphics displays in 
general, and the display and equipment used in the thesis experiments. 

Chapter 4 discusses current and projected applications of three-dimensional 
perspective displays in underwater visualization. 

Chapter 5 describes in detail the conduct of seven tests including restricting the field 
of view, grids to provide depth and angular cues, color schemes to provide altitude cues 
versus depth of field, direct range information, and slowing of the graphic display update 
rate. 

Chapter 6 presents the results of the seven tests including operator comments and 
author/test monitor observations. 

Chapter 7 summarizes the results of the experiments, discusses conclusions, and 


offers recommendations for future work. 
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Chapter 2 


Human Factors Engineering 


2.1 Objectives and Doctrines 


Human factors engineering 1s the study of people and how equipment designs affect 
them. The two major objectives of human factors engineering are to improve efficiency 
and effectiveness and to promote key elements such as safety, satisfaction, and stress 
reduction. Six doctrines that form a set of human factors commandments are [S-1]: 

e Design with the user in mind. 

е Recognize differences in capabilities and limitations of people and their design 

implications. 

e Design influences human behavior. 

e Design process includes empirical data and evaluation. 

e Reliance on scientific methods and objective data. 


e Commitment to system orientation. 


2.2 Methodology Issues 


Experimental research methods test the effects of variables on behavior. Of prime 


interest in this work is the behavior "efficiency of operator performance". The method 
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selected needed to accurately measure this efficiency. Eight important issues associated 
with the choice of methodology are [M-1]: 

e Effectiveness. 

e Ease of use. 

e Cost. 

e Flexibility. 

e Range. 

e Validity. 

e Reliability. 

e Objectivity. 

No approach was best for all the issues above. Among them effectiveness, 
validity, reliability, and objectivity were deemed most critical for this thesis. Validity, in 
particular, was enhanced by input from experienced ROV pilots and operational support 
group Е апе. These inputs served to more realistically match the simulation to 


operating conditions. 


2.3 Research Criteria 


Research criteria can be divided, with inevitable overlap, into the areas of system- 
descriptive criteria, task-performance criteria, and human criteria [M-1]. The criteria 
chosen for analysis in this work was task performance, specifically performance time. 
Human criteria cannot be ignored, however, and provide insight in the form of subjective 
comments by the ee both during and after each test, and observations by the test 


monitor/author. 
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2.4 Visual-Coding 


Color is a prime visual-coding technique. It has been frequently used in 3-D 
underwater visualization to provide depth/relief cueing in the form of a continuous color 
scale versus depth. In this work two discrete color sets were chosen for testing. Both the 
6- and 21-color sets were derived from the red, green, and blue (RGB) color model. 
Figure 2.1 shows the RGB cube [F-1]. The 6-color set was selected by following a path 
from corner to corner in the following order: red, yellow, green, cyan, blue, and magenta. 
The 21-color set followed the same path, selecting colors at each 0.25-unit path-length 
increment. 

The designation of specific color names was complicated by the existence of a 
variety of descriptive systems. Among the more notable were RGB, HSB, Munsell, 
ISCC-NBS, YIQ, CIE, and UCL [B-1, F-1, K-1]. Target color names were derived 
starting with the basic six colors of the RGB color set (identical to the six basic hues of the 
HSB color set): red, yellow, green, cyan, blue, and magenta [F-1]. The further 
subdivisions were named in accordance with the spirit of the Color Naming System 
(CNS), which was developed to simplify color descriptions [B-1]. The 21 color names 
derived were used for the 1-, 6-, and 21-color sets. Table 2.1 lists the colors by name and 
includes RGB and HSB descriptions. Conversions between most color sets were available 


and easily done with a computer program [F-1]. 


2.5 Control and Tracking Performance 


Controls transmit discrete or continuous signals to a system to produce a desired 
response. Important factors in control design include ease of identification, size, control- 


response ratio, resistance, lag, deadspace, backlash, location, and compatibility [S-1]. 
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Red = (1, 0, 0) Yellow = (1, 1, 0) 
Figure 2.1 The RGB (red, green, and blue) cube [F-1] 


Table 2.1 Color set RGB to HSB conversion 
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Deadspace refers to a null established about the control's normal position in which control 
device motion results in no signal output. Backlash is deadspace at all control positions. 
The control mechanism may have no resistance (free-positioning or pure-displacement 
control), no displacement (suff stick or pure-force control), or lie in the middle (which we 
term a mixture control). 
The control order of a system refers to the relationship between the control 
mechanism and the system output. Orders of control include [S-1]: 
e position (zero-order) control - where the signal controls the output directly. 
e velocity (first-order) control - where the signal controls the rate of change of 
position. 
e acceleration (second-order) control - where the signal controls the rate of change 
of velocity. 
e higher-order control - where the signal controls the rate of change of acceleration 
or other higher order term. 
The system in these tests used second-order control of thrust, equivalent to acceleration, in 
each of the four degrees of freedom (see Section 5.3). 
The problem presented to the operator was a compensatory tracking problem with 
time lag [S-1]. The time lag was composed of three parts: 
e response lag - the time required for the operator to choose and execute a 
response. 
e control system lag - the time required for the system to respond to commands 
including vehicle motion simulation. 
e display system lag - the time required for graphics calculations and display 
update. 


In each case the time lag increased tracking error. 
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2.6 Test and Evaluation 


Independent variables (IVs) are those parameters controlled by the experimenter. 
Dependent variables (DVs) are measured parameters affected by the IVs. They are the 
same as the research criteria discussed in Section 2.3. 

Evaluation of test results must consider test subjects, criteria, and experimental 
procedures [S-1]. Subjects should be representative of projected users. Dependent 
variables (criteria) must be related to system operational use. Experimental procedures and 


controls should give repeatable results. 
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Chapter 3 


Three-Dimensional Computer Graphics Displays 


3.1 True 3-D Displays 


Technology stands at the threshold of true three-dimensional displays. Available 
implementations display depth instead of simply providing depth cueing. Two such 
systems are SpaceGraph™ by Bolt, Beranek, and Newman (BBN) Laboratories 
Incorporated of Cambridge, Massachusetts and OmniView™ by Texas Instruments (TI) of 
Dallas, Texas. 

The SpaceGraph™ system is based on reflecting a point plotted (not raster scanned) 
image in a vibrating mirror. The technique is termed the varifocal mirror. As the reflecting 
surface vibrates it repeatedly changes shape from concave to convex, causing a large 
change in focal length. When the image update rate is synchronized with the mirror 
vibration, a true 3-D image is created in space. For many data-set analyses the superior 
pattern-recognition capability of the human brain is enhanced by this type of display. This 
system is limited in viewing angle, display update rate, and to simple monochrome images. 
[F-1, T-1, S-2] 

The OmniView™ system is based on laser scanning of a paraboloidal, rapidly 
rotating surface. During rotation, the scanning laser beams reflections form the 3-D 
image. Advantages include 360? horizontal viewing and total wrap-around parallax. This 


system has no hidden surface removal and cannot display solid surfaces. [E-1] 
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Holography is another promising technology for true 3-D viewing. This work is 
based on the splitting of a laser beam into two parts, illumination of objects by one of the 
split beams, and their subsequent recombination using the second beam as a reference. 
Limitations typically include small size and restricted viewing angle, lack of parallax, and 
high cost. [E-1, F-1] 

Head-mounted displays have offered tremendous potential since their birth in the 
late 1960s [S-3, S-4]. The helmeted user enjoys freedom of movement in a virtual 3-D 
world projected in real time for separate viewing by the each eye. Technical problems have 
severely restricted the development of this technology since its inception [F-1]. 

An inherent limitation of true three-dimensional viewing is the tremendous quantity 
of data and associated data processing, particularly critical for real-time display. Expenses 
for computation rise dramatically as realistic, complex 3-D imagery is approached and 
parallel-processing supercomputers become a necessity. Significant effort and funding are 
being funnelled into this cutting-edge technology, but it is not likely to be available to the 


field scientist before the late 1990s. 


3.2 Simulated 3-D Displays 


In the absence of true 3-D displays several methods are employed to simulate three- 
dimensional viewing using two-dimensional CRTs. These methods are designed to 
counter the information loss inherent in projecting 3-D information on a 2-D surface such 
as a computer screen. Techniques used include those for line drawings and shaded images, 
as well as dynamics, stereopsis, and head-motion parallax. 

Line drawing techniques include the use of various projections, depth cueing with 
intensity, depth clipping, and hidden-line removal [F-1]. Among the projections, the 


perspective offers two significant depth cues: decreasing size of objects with depth and the 
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convergence of parallel lines to their vanishing points. Depth cueing with intensity draws 
the more distant objects at a lower intensity. The result is a strong depth cue similar to that 
observed when viewing a scene in a light fog. Depth clipping limits the view by preventing 
the drawing of lines not within the designated limits of the front or back clipping planes. 
This effect is used to limit the depth of field. Hidden line removal calculates the lines that 
should not be visible based on the view and does not draw them. 

Shaded-image techniques include hidden-surface removal, control of illumination 
and shading, various shading interpolation methods, and the use of texture, shadows, 
transparency, and reflection. 

Dynamics refers to the added information available when a sequence of views is 
shown instead of a static view. The primary depth cue is provided by the kinetic depth 
effect, the lower velocity of nearby objects versus those in the background. 

Stereopsis is a depth cue taking advantage of the binocular disparity between human 
eyes [F-1]. The ability of the human brain to fuse two slightly differing views of a scene 
and extract depth information is a powerful depth cue. 

Head-motion parallax uses the changing view as occurs in real life when the head is 


moved side to side. 


3.3 The Marquest System 


Several important components of the test equipment hardware and software were 
provided by the Marquest Group, Inc. of Bourne, Massachusetts. They include the 
joystick module and associated software, an Nth Engine/550™ Display Controller, and 
HOOPS™ graphics routines and HY DRA™ rendering software for the Nth Engine/550™, 
They are components of a commercial ROV control system. 

The joystick module is a standardized control device for ROVs constructed by the 


Marquest Group, Inc. using off-the-shelf components. Included are a trackball, bias and 
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gain potentiometers, nine push-button switches, and а 3-axis joystick. Only the joystick 
was used in the testing. Measurement Systems, Inc. of Norwalk, Connecticut, built the 
industry standard, heavy duty, 3-axis (x, y, 0) joystick with mixture control and deadspace 
on each of the three axes. A thumb switch provided the fourth axis of freedom (z) with 
pure force control. Another joystick available from Measurement Systems, Inc., but not 
used here, incorporated the fourth axis of freedom through lifting/depressing the joystick 
vertically with mixture Control. Joystick use is discussed in Chapters 6 and 7. 

The Nth Engine/550™ Display Controller is a 3-D graphics processor/controller 
built by Nth Graphics of Austin, Texas. The controller has 1280 x 1024 resolution and 
operates at 20 MIPS and 3 MFLOPS. The board set includes an NthTV™ converter 
module, which provides a standard commercial television format (NTSC) tap for direct, 
real-time recording of the video display on a video cassette recorder (VCR). 

HOOPS™ from Ithaca Software of Ithaca, New York, formed an interactive three- 
dimensional graphics library. This software controlled the Nth Engine/550™. 

HYDRA™ rendering software, another product of Nth Graphics, was used to 
convert targets and grids drawn in AutoCAD® Release 10 to the required format for the 
HOOPS graphics routines. A helpful feature is the ability to quickly conduct a visual check 


of targets and grids. 


3.4 The Display and the Monitor 


Planar geometric projections fall into two categories: perspective and parallel. 
Characteristics which distinguish a perspective projection from a parallel projection are 
convergence of parallel lines, size diminution, and nonuniform foreshortening [C-1]. The 
primary advantage of a perspective projection is the natural appearance of objects seen by 


the eye. It was also the standard video camera projection. For these reasons a three-point 
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perspective projection was chosen for the display [C-1]. The graphics were shown on a 
Sony model GDM-1950 19-inch color monitor with 1280 x 1024 pixel resolution. 

Of the previously discussed techniques to provide three dimensionality, depth 
cueing in particular, many were not available in the experimental system for three reasons: 
the tests used a perspective projection, the targets were drawn in wireframe mode, and 
limitations of the Nth Graphics boards and software running the display. Depth cueing 
was provided by target relative size (because they were all identical in dimension), the 
convergence of parallel lines (emphasized by the cubic wireframe targets), and the kinetic 
depth effect. 

An NEC VGA color monitor (with blue background) provided the following: 

e display update indicated by the word "Busy..” appearing in white letters in a red 

box. 

e most recent target hit indicated by the words "OBJECT #(target number) HIT" 

appearing in white letters in a red box. 

e test completion indicated by the words "MISSION COMPLETE" appearing in 

white letters in a red box. 
e range in meters from the vehicle (modelled as a point) to the center of the next 
target indicated by the symbol "R = (range)" appearing in white letters in a 
red box. This information was provided to the operator only during test 6. 

e world system coordinates (x, y, and z) and heading of the vehicle in white 
numerals. Titles (xpos, ypos, zpos, and hdg) were shown beside the 
coordinates/heading in white letters. 

e vehicle system velocities (X, y, ż , and 0 rate) in white numerals. The titles 

(xvel, yvel, zvel, and rvel) shown in black letters above the velocities were 
individually accented with small red boxes if their magnitudes exceeded 
0.05 m/s or 0.05 гад/5. 


Figure 3.1 shows the monitor screen with range information during test 6 . 
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Figure 3.1 Monitor screen during test 6 
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Chapter 4 


Application of the 3-D Perspective Display 


4.1 Ocean Environmental Sensing 


Early sonar development was driven by military needs. Crude processing and 
display techniques limited these systems to deterministic methods with severely restricted 
dynamic range. Inherent in these limitations were a disregard for inaccuracies such as 
angular and range resolution, variations in the speed of sound and the sound ray paths, and 
Navigation errors in position, attitude, depth and motion. The current state of technology 
overcomes many of these restrictions. Modern computers and software graphics packages 
allow real-time processing and display of more advanced models of the ocean environment. 
New commercial sonars provide more accurate information from a variety of sampling 
techniques including swath mapping, sidescan, and upward-looking sonars, in addition to 
the more traditional forward-looking and bathymetric sounding versions. Innovative 
electronic instruments provide high-accuracy navigational data. Recent trends have resulted 
in additional technologies being adapted for underwater work. Video and electronic still 


cameras, and scanning lasers are among these supplementary sensors. 
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4.2 Multisensor Modeling 


To effectively combine the various sensor modalities into a single model of the 
ocean environment, we must first accept the premise that no measurement is exact. Any 
information gathered will have an associated probability distribution, which is itself not 
known deterministically. The ocean model can be built incorporating these uncertainties as 
a stochastic model. This approach is especially effective when combining information from 
different sources with their differing ranges, resolutions, and data types. At any time, a 
deterministic model can be extracted from the current probabilistic one. Using this method, 
a more accurate deterministic model is obtained which accounts for the many and varied 
errors inherent in measurements. 

Initially, the ocean volume of interest is divided into a 3-D grid of cubic volume 
elements, or voxels. The size of the voxels is dependent upon the desired model resolution 
as well as system hardware and software limitations for real-time processing. Each voxel 
is assigned a "feature" vector incorporating information such as the time, the position of the 
voxel in the coordinate space, the measured characteristics of interest (features), and a 
probability or confidence value for each feature. As new information is collected, older 
data is not discarded. Feature vectors are incremented using a data merging technique 
called stochastic backprojection [S-5, S-6]. This data merging technique is an adaptation of 
the backprojection and summation method used to reconstruct images. The method is 
enhanced to include probabilistic uncertainty and allow stepwise solutions for real-time 
processing. Use of these techniques allows combination of the redundant feature data from 


all available sources, improving the model resolution and certainty. 
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4.3 The Deep Submergence Laboratory System 


The Deep Submergence Laboratory of the Woods Hole Oceanographic Institute 
employs a variety of sensors and platforms. A typical cruise involves flying equipment to a 
proximate port and loading out a chartered mother vessel. At the research site, a large area 
survey is conducted with a towed 120-kHz split-beam sonar system. This survey provides 
medium resolution imaging with a swath width of 200-300 meters at a tow speed of 6-10 
knots, and allows items of interest to be identified for subsequent revisiting with higher 
resolution equipment. Items of interest are reinspected using the tethered Jason ROV 
system. Jason is typically equipped with a SHARPS short-baseline navigation system and 
a sensor package including a 200-kHz sidescan sonar, a 300-kHz forward-scanning sonar, 
a video camera, an electronic still camera, and a laser rangefinder. Figure 4.1 shows the 
range and resolution overlaps between sensors in the current suite. 

The Jason pilot has several sources of information available. If within visual range, 
the video camera and vehicle lighting system provide a natural, perspective view with 
extremely high bandwidth. This visual range is often restricted to a meter or less due to 
suspended sediment in the water, and is usually less than about 10 meters. Beyond visual 
range the forward-scanning sonar, the 200-kHz sidescan sonar, the laser rangefinder, and 
the SHARPS navigation system provide data to several displays with differing formats. 
The pilot navigates using a specially designed 2-D auxiliary display with limited bandwidth 
until within visual range [S-7]. 

This current method taxes the pilot, whose task is complicated by [S-7]: 

е low visibility, reducing video camera range and safe working speeds. 

e the narrow field of view of the video camera (approximately 40°). 


e the monocular vision of the video camera. 


optical distortion of the video camera. 


the difficulty of interpreting sonar presentation screens. 
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e the transient nature of the displayed information, requiring the pilot to mentally 

integrate the overall picture. 

e the inability to access the high-level, real-time environmental model. 

To enhance ROV piloting efficiency, a real-time, 3-D display is envisioned. The 
display will access the high-level environmental model and present the user with a view 
closely resembling that of the video camera. Because low or medium resolution data are 
available prior to the ROV entering the water, a rough perspective-view display can be 
constructed at any time for use by the pilot on the screen previously reserved for video 
camera information. Such a perspective-view display is expected to be dominated in the 
foreground by high-resolution video data, gradually transitioning to dominance by other 


sensors as the limits of video range are encountered. 


4.4 Targeted Human Factors 


Among the myriad of potential display enhancements to improve the proposed 3-D 
perspective display are: 

e altering the displayed field of view. 

e providing a screen grid. 

e displaying current range to the target of interest. 

e use of a vertical color scheme. 

e controlling the display update rate. 

Narrowing the displayed field of view reduces the necessary graphics calculations 
as the square of the reduction factor. This gain must be balanced against the loss of 


information to the user. 
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Providing a 2-D or 3-D screen grid can improve the user's angular perception, 
provide a cross-hair effect to ease direction finding and, with the 3-D version, provide 
important depth cues. 

Displaying current range to the target of interest can reduce the time to locate the 
target. This information is available in the environmental model. 

Vertical color schemes are frequently used to provide redundant relief cueing for 
underwater and land feature presentations such as charts and maps. They can be effective 
in giving altitude cues for objects 1n the background of perspective displays. 

Display update rate is limited by the data-collection cycle time of the sensors, data 
processing time, and graphics calculation/display time. Compromises can be made to 
shorten the update rate at the cost of information quality and quantity. 

This thesis attempts to quantify the effects of the task-related independent variables 
listed above. The dependent variables for the experiments are performance time, subjective 


comments by the operators, and observations by the test monitor/author. 
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Chapter 5 


Testing Display Effectiveness 


5.1 Equipment 


Three system hardware components were used directly by the operator: a computer 
monitor, a computer display, and a joystick. Other equipment transparent to the operator's 
task included a 386 computer with keyboard and mouse, which ran the simulation and 


recorded the data, and various power supply and communications cabling. 


5.2 Data Point Courses 


Nineteen data courses (A-R, X) were used in testing. Courses consisted of 25 
sequentially numbered data points (Pi) in world coordinate system three space (x, y, z) 
with two constraints: 

(1) Consecutive points were separated by distance s — V[Ax]2+[Ay]2+[Az]2, where 

s was a constant equal to 5 meters. 
(2) A square window of given total angular width and height B, with apex at point 
Р, and opening in the direction of a line from Pj.1 through Pj, contained 
point Pi+1- 
Courses A, B, C and J, K, L used a 15”-square window, courses D, E, F and M, №, О 
used a 30° window, and courses G, H, I and P, Q, R used a 60° window. Course X, the 


training/familiarization course, used a 15° window. Additionally, nine of the courses were 
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mirror images of each other in the world coordinate system y/z plane. Mirror pairs were 
A/J, B/K, C/L, D/M, E/N, F/O, G/P, H/Q and I/R. Figure 5.1 and table 5.1 summarize 
the above information. 

Visualization of each data point was provided in the form of a cubic wireframe 
target one meter on a side with the invisible data point at its center. Target faces were 
parallel to the planes formed by the world coordinate system axes. Target color was blue 
except for tests 5 and 6 as discussed in section 5.7. Each target was numbered with white 
numerals (1-25) corresponding to the sequential number of the data point at its center. A 
target "hit" corresponded to closing the range to the target center/data point to 0.5 meters or 
less from any direction. Appendix C discusses the details of target design. Appendix D 
contains course layout data and photographs of the courses as viewed by the operator. 

Random generation of the data points for each course began with establishment of 
two coordinate systems, the world and the vehicle. Each system consisted of three 
mutually perpendicular axes x, y, and z, forming a right-hand coordinate system. The 
world system was fixed in space. In ıt, three angles were defined: о), @, and K for positive 
rotation about the x, y and z axes respectively. Positive rotation was defined such that 
positive © (Q, K) rotated the positive y (z, x) axis into the positive z (x, y) axis through 90°. 
In the vehicle system, the operator was assumed to be looking down the positive y axis of 
the system at all times with the positive x axis to the right. Positive rotation was defined so 
positive 0 rotated the positive y axis into the positive x axis through 90°. In this way, © 
had the same sense as true heading. Figures 5.2 and 5.3 depict these systems. For a 
specified window width D, sets of two angles o and K were randomly selected from the 
interval [-B/2, D/2]. This process insured that the data points met constraint (2) above. 

Based on the above coordinate system definitions, rotations of the vehicle system 
were broken down into individual rotations of œ, Q, and K. Orthogonal orientation 


matrices were then defined for the decomposition as follows [M-2]: 
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My = 0 соб Ф сіп 0 
0 -sin œ cos © 
cos Q 0 -sin Q 
Мо = 0 1 0 
sin Ọ 0 COS Q 
cos K sin K 0 
Мк= | -sin K cos K 0 
0 0 1 


The most general orientation matrix M for any combination of these three rotations 


was then [M-2]: 


М =МоМоМк 
COS Q COSK cos O sin K + sin Q sin @ cos K 
= | -cospsink cos ®© сов к - sin O sin Q sin K 
sin Ọ -SIN © COS Q 
sin © SIN K - COS © SIN Q cos K 
510 Ф со5 К + Cos © SinQ sin K 
cos © cos Q 


In our application pitch (œ) and yaw (xK) were allowed to vary while roll (Q) was 


not. The general case was then specialized by setting Q = O to obtain: 


cos K cos @ sin K sin Ф sin K 
L=Mọ=0 = | -sin K cos (WM COS K sin © COS K 
0 -sin O cos w 


The matrix Lj defined the look direction from the point Pj to the next point Pj; in 


the vehicle coordinate system. The matrix (L;)" defined the transformation from the vehicle 
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coordinate system to that of the world. The origin point was taken to be Pg = [ 0,0,0 11. 
The first data point was taken to be P1 — s[ 0,1,0 ШЕ where s was the constant described in 
constraint (1). The value (0, 0) was assigned to the set (©, x) at x = 0 yielding Lọ = 13,3, 
the three-dimensional identity matrix, and successive Lis were generated using the above 
formula for matrix L and the j-th set of values (w, K). As a result of these specifications 
EE = I3x3 and successive (1) 5 were calculated from the equation (Lisi) = ШЕ 


Successive data points were then generated using Р 1 = Р, -- s(L;)"L; [0,1,0 ]T as the 


recursive algorithm. 


5.3 Modeling the Jason Vehicle 


The vehicle model was a dynamic simulation of the DSL ROV Jason. The model 
exhibited four degrees of freedom: x (left/right), y (forward/reverse), z (up/down), and @ 
(CW/CCW). Hydrodynamic drag was assumed to have quadratic and linear (damping) 


components. Balancing forces yielded the equation (shown for the x component) : 


(M + Ma) 
(М М.) Ў + а eh 
Tmax 
where, 
M was the vehicle's mass, М, was the vehicle's added mass, and (M + Ма) 


was termed the vehicle's extended mass. 


Tmax Was a constant (2.0 s) defining the magnitude of the linear drag. 
Cp was the drag coefficient. 
Т was the maximum thrust. 
Table 5.2 summarizes the model parameters use in the experiment. Nonlinear 


thruster dynamics were not included in the vehicle simulation model. 
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The square window actually defined a portion of the 






surface of a sphere because point БҮРІ was required 


to be exactly distance s from point Pi. 






= Pj4] 


window 
3-1 
Figure 5.1 Visualization of data point course constraints 


Table 5.1 Data point course relationships 
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Table 5.2 Dynamic vehicle simulation parameters 


P Extended Mass | Drag Coefficient 
(kg) (kg/m or kg-m) (m/s or rad/s) 
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Figure 5.2 The world coordinate system 


Front => 


Figure 5.3 The vehicle coordinate system 
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5.4 Operators and Training 


The same four operators were used in each of the seven tests. The author was not 


an Operator. Operator profiles were: 


Operator I was a 22 year old, male Naval Officer and graduate student with 
uncorrected full visual acuity and normal color vision. This right-handed subject 
had frequently played video games including flight simulators. He was familiar 


with 3-D perspective computer displays prior to the experiment. 


Operator II was a 24 year old, male graduate student with uncorrected full visual 
acuity and normal color vision. This left-handed subject had little exposure to video 
games such as flight simulators. He was familiar with 3-D perspective computer 
displays prior to the experiment due to practical experience with computer systems 


as an undergraduate Computer Science major. 


Operator III was a 34 year old, male Naval Officer and graduate student with 
uncorrected full visual acuity and normal color vision. This right-handed subject 
had some exposure to video games such as flight simulators. He was familiar with 


3-D perspective computer displays prior to the experiment. 


Operator IV was a 32 year old male with a Master's Degree, uncorrected full visual 
acuity, and normal color vision. This right-handed subject had little exposure to 
video games such as flight simulators. He was not familiar with 3-D perspective 


computer displays prior to the experiment. 


Prior to each testing session, operators were briefed in the following areas: 
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general, simulation, and hardware. The general topics included purpose of thesis and 
experimental approach, time measurement, and operator errors. Simulation topics included 
target and course design as well as vehicle modeling. Hardware subjects discussed impor- 
tant features of the computer monitor, the computer display, and the joystick. Additionally, 
a maximum of 15 minutes was allotted for familiarity training with a 60°, 30°, or 15° field 
of view, as appropriate to the test, using training course X. 

The tests were conducted in a total of four separate sittings to reduce operator 
fatigue. Test 1 (see number designations below) was conducted alone. The remaining 


tests were conducted in pairs in the following order: 2/3, 4/5, and 6/7. 


5.5 Test 1 - Field of View 


Testing for field of view measured changes in the operator's spatial performance 


versus the displayed field of view. 


Test 1 description: All targets were colored blue. The operators drove the simulated 
vehicle point to point over a course of 23 data points spaced in x, y, and z. Operators 
completed nine courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
data from point to point. Measured fields of view were 15°, 30°, and 60°. The display 
update period was fixed at two seconds. Nine combinations of FOV and courses were 


used with each of the four operators. 


5.6 Tests 2 and 3 - Gridding 


Testing for gridding measured the improvement in operator performance when 
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presented with a visual reference grid in the display. 


Test 2 description: All targets were colored blue. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
data from point to point. The field of view was fixed at 60° and the display update period at 


two seconds. A two-dimensional (2-D) grid was superimposed on the visual display. 


Test 3 description: All targets were colored blue. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
data from point to point. The field of view was fixed at 60° and the display update period at 


two seconds. A three-dimensional (3-D) grid was superimposed on the visual display. 


5.7 Tests 4 and 5 - Vertical Color Schemes 


Testing for color scheme measured the improvement in operator performance using 
a fixed, vertical-relief color scheme for target shading versus single-color targets. The 6- 
color set provided 12 meter vertical color swaths while the 21-color set provided three 


meter vertical color swaths over the range of interest. Table 5.3 lists the color sets. 


Test 4 description: Targets were shaded in color according to height z above ocean floor. 
The color scheme was divided into six discrete shades. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 


completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
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data from point to point. The field of view was fixed at 60° and the display update period at 


two seconds. 


Test 5 description: Targets were shaded in color according to height z above ocean floor. 
The color scheme was divided into 21 discrete shades. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
data from point to point. The field of view was fixed at 60° and the display update period at 


two seconds. 


5.8 Test 6 - Heads Up Range 


Testing for range measured the improvement in operator performance when 


presented with continuously revised range to target center information. 


Test 6 description: All targets were colored blue. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
data from point to point. The field of view was fixed at 60° and the display update period at 


two seconds. Continuously revised range to target center was provided to the operator. 


5.9 Test 7 - Display Update Rate 


Testing for display update rate measured the degradation in operator performance 


associated with reduced visual information feedback. 
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Test 7 description: All targets were colored blue. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
data from point to point. The field of view was fixed at 60°. The display update period 


was slowed to four seconds. 
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Chapter 6 


Testing Results 


6.1 General 


The first step in the analysis was to convert the logged target hit times into time 
differences between successive targets. Appendix E lists the resultant time differences. 
The second step was normalize the raw data from Appendix E by removing the effects 
properly attributed to variations in the level of course difficulty related to the Jason vehicle 
simulation model. This procedure divided each time difference (in seconds) by the best 
time (in seconds) that could be made between the associated targets based on the vehicle 
simulation. Appendix D discusses the method of arriving at this best time between targets 
and lists those times in table form. Appendix F contains the normalized, dimensionless 
values. The third step was to estimate the operators’ learning curves to standardize results 
between tests. These results were then analyzed directly to assess the effectiveness of the 


tested display enhancements. 


6.2 Operator Learning Curves 


Operator learning reflected a number of factors including increasing familiarity with 
test equipment, improving knowledge of test procedures, and gradual easing of pretest 


anxiety, as well as modified strategies for driving the simulated vehicle. Estimation of the 
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operators' learning curves involved four assumptions. 

e The learning curves had positive slope at all times. This meant that the operators 
were constantly improving with no loss of knowledge or acquired skills 
between testing sessions. 

e The learning curves were smooth functions. This prohibited the consideration of 
learning plateaus during which operators would show no improvement. 

e The learning curves were exponential functions. 

e Fora specified field of view, operator learning was reflected primarily in the 
standard deviation of the normalized course times (a smoothness measure). 
The effectiveness of display enhancements was reflected primarily in the 
total course times (a measure of speed). 

As an additional consideration, test 1 was designed to force the operators to use a variety of 
strategies, particularly with the reduced fields of view, in an attempt to ensure maximum 
learning in minimum time. 

Each operator drove 21 courses with a 60° field of view, three for each of the seven 

tests. The procedure below was applied to the normalized data for each operator to estimate 


his 60° FOV learning curve. 


e The standard deviations were entered into a 3 x 7 matrix. Each row represented 
a specific window width B as discussed in section 5.2 (row 1 = 15°, row 2 
= 30°, and row 3 = 60°). Each column corresponded to a given test (column 


1 = test 1, column 2 = test 2, etc.). 


e The standard deviations for the three window widths for each test were summed 
and entered into a row vector of seven elements. Each column continued to 
represent a given test as above. This step yielded the summed standard 


deviations (SSD) shown in table 6.1. 


e The row vector elements were plotted with summed standard deviations on the 


vertical axis versus test number on the horizontal axis. 
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e An exponential curve using the method of least squares was fitted to the plot by 
linearizing the exponential form through taking logarithms [G-1]. This 
defined the operator's learning curve. Appendix G shows the graphed 
operators’ learning curves. 

Once the operators’ learning curves were estimated, correction factors to allow 

results between tests to be compared were derived using the following procedure. 

e Estimated standard deviation values were extracted from the fitted exponential 
curve using regression techniques. This step produced the extracted values 
(EV) listed in table 6.1. 

e Standardization factors (SF) were formed by dividing all EVs by the first (and 
largest) value. The EVs are listed in table 6.1. 

е The 84 mean values of the normalized 60° FOV course times listed in Appendix F 
were multiplied by the associated SF. This step resulted in the learning- 
corrected, normalized mean times (LCNMT) listed in table 6.2. 

e For each test, the LCNMTs for all operators were summed. The mean and 
standard deviation of each LCNMT sum were found, completing the data 
manipulation. 

Figure 6.1 shows the plotted means and standard deviations of the normalized mean times 
without correction for individual operator learning curves. Figure 6.2 shows the means 
and standard deviations of the LCNMT sum plotted for analysis. 

The learning curve for the different fields of view could not be estimated using the 

same techniques. It was assumed that the individual operator's 60° FOV learning curve 
was a good approximation and the first three SFs from table 6.1 for each operator were 


applied to the data. 
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Table 6.1 Summed standard deviations (SSD), extracted values (EV), and 


standardization factors (SF) 
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6.3 Field of View 


The three fields of view used in testing were related by inverse powers of two (60°, 
30°, and 15°). Plotted means and standard deviations of the learning-curve-corrected data 
versus FOV scaled by inverse powers of two were virtually linear and clearly demonstrated 
the parabolic relationship between the mean run time and the FOV. Fitting a parabola to the 


data showed that: 


normalized mean run time = 10.399 - 0.200 * FOV + 1.350 * 10-3 * FOV2. 


6.4 Gridding 


The 2-D and 3-D grids improved operator performance to the same degree (34.64 
and 34.92 respectively). The 2-D grid was simple and straightforward to employ. The 3- 
D grid conveyed more information, but cluttered the display and was harder to interpret. 
The primary benefit of the 2-D display was the "cross-hairs" effect of the grid, which aided 
steering effectiveness. This effect was less pronounced with the 3-D grid. Hidden-line 
removal, particularly for a 3-D grid, could provide strong visual range cues in the vicinity 


of a target. 


6.5 Vertical Color Schemes 


The 6-color scheme and the 21-color scheme both provided improved operator 
performance. The 6-color scheme proved to be slightly better (32.34 versus 34.68). This 


effect was attributed to its simplicity and the high-contrast, easily distinguished colors. The 


52 





Table 6.3 Learning-corrected, normalized mean times (varying FOV) 
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primary benefit of the color schemes was improved spatial perception of the display due to 


the combined effects of depth and altitude cues. These two cues were strongly interrelated. 


6.6 Heads-Up Range 


A surprising result was the degradation of operator performance when supplied 
with range to the target center. Although the range information proved valuable if a target 
was not visible, was close, or had been missed, this was more than offset by negative 
factors. Primary problems included ignoring the visual information of the display and the 
“perigee effect,” where a control action that caused the range to initially decrease would not 
necessarily result in a hit. . As the vehicle's trajectory passed perigee with the target's 
center, the range increased, frequently confusing the operator. No individual operator's 


performance improved with range information. 


6.7 Display Update Rate 


As anticipated, slowing the graphics display update rate by a factor of two hurt 
operator performance. The magnitude of this effect, slowing the normalized run time by a 
factor of 1.3, was smaller than anticipated. This had significant implications for field 
applications, where a longer update period would allow the use of less powerful computer 
equipment or more robust programming. The loss of the operator's ability to mentally 
integrate the visual information from the display and use kinetic depth-effect cues was 
mitigated by the more conservative driving approach adopted by the operators. All the 


Operators demonstrated this slowing effect with update rate. 
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6.8 Operator Comments 


Relevant operator comments were broken down into the categories of field of view, 
grids, color sets, range data, update rate, test equipment, test design, and strategy 
(operator number is included in parentheses after comments). 

e Field of View- Narrowing the field of view placed a premium on 

maintaining visual contact with the targets (II). 

e Grids - The "cross-hairs" effect aided steering effectiveness (II, IV). 

Grids provided relative size information (II). 

The 3-D grid was more intuitive and effective than the 2-D 
grid (II). 

The 3-D grid cluttered the display (IV). 

The grids were not particularly helpful (III). 

Hidden-line removal for grids could provide strong visual 
range cues in the immediate vicinity of a target by 
showing the target "poking through" the grid (I, IV). 

The black grids were sometimes confused with the blue 
targets when the display was cluttered (IV). 

The operator's need for a grid was reduced by improvements 
in his spatial perception (IV). 

e Color Sets - Color provided spatial visualization cues, especially with 

overlapping targets (all). 

The 21-color set provided significantly more information and 
was more effective than the 6-color set (II, IV). 

Color information was far superior to gridding (III). 

Target course altitude slope information was available from 


the rate of change of target colors (III). 


55 








| 


A sliding color scale with the vehicle always at scale center 
could provide improved relative altitude cues (III). 
e Range Data - Range information proved valuable if a target was not 
visible, was close, or had been missed (all). 
Range information was ignored if a target was in view (ID. 
Range information provided for smoother driving near 
targets and eased hitting targets off-center (II, IV). 
The operator perceived that targets were "hit" from a greater 
distance with range information available (IIT). 
e Update Rate - Operators drove more conservatively with the slower update 
rate (II, IIT). 
Update rate was more important as the range decreased (III). 
Vehicle alignment with the target trend was more important 
with the slower update rate due to the time lag in 
update rate while rotating (IV). 
Operator workload increased with slower update rate (I). 
The slow update rate was not as difficult as anticipated (IV). 
e Test Equipment- — The joystick was very tiring to use, particularly the z-axis 
pure-force control (III). 
Fine z-axis movement was difficult due to the stiff pure- 
force control (1I, III). 
Axis movement with the joystick was not compatible with 
operator expectations (I, III). 
Presentation of all information on one screen could ease 
operator workload (all). 
e Test Design - Target drawing order proved confusing (I, II, III). 


The display graphics can be cluttered, particularly with a 
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e Strategy - 


narrow field of view (I, II, IIT). 

The simulated vehicle lacked an effective altitude search 
capability (I, III). 

Operator efficiency was reduced by the fixed 2-D target 
numbers when approaching targets from the side 
(all). 

Improved heading perception was provided by the fixed 2-D 
target numbers (II). 

Operator comfort with vehicle rotation improved as testing 
progressed (I, III). 

Operators needed to plan more than one target in advance 
and, when possible, kept them on the display (II, 
ПІ). 

Rotating the vehicle to line up along the target heading trend 
reduced the time between targets (II, III). 

Knowledge of the course trend aided target identification 
(IV). 


Driving while backing up was counterintuitive (III). 


6.9 Test-Monitor Observations 


Test-monitor observations not previously listed under operator comments were 


broken down into the categories of color sets, range data, update rate, test equipment, test 


design, strategy, and tendencies. 


e Color Sets - 


Some colors were weak in contrast with the background 
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e Range Data - 


Update Rate - 


Test Equipment - 


Test Design - 


Strategy - 


dark gray. Reddish Orange and Turquoise were 


most difficult for the operators to discern. 


Range information sometimes handicapped the operators by 


providing too much data, causing them to ignore the 
display visual information. Another detrimental 
effect was that a control action that caused the 

range to initially decrease would not necessarily 
result in a hit. As the vehicle's trajectory passed 
perigee with the target's center, the range would 
increase. This effect fooled all operators initially, 
particularly when moving the vehicle along more 


than one axis simultaneously. 


The four-second update period significantly degraded the 


operator's ability to mentally integrate the display 


information and use kinetic depth-effect cues. 


Stiffness of the joystick resulted in unorthodox grips being 


employed, many with both hands. 


An artifact of the wireframe target design was that the 


simulated vehicle could be close to or within the 
target without any portion of the target being visible 
onthe display. This is an unlikely scenario in the 


field. 


Backing up to keep the previous target in view while looking 


for the next target was an effective approach when 


presented with a limited field of view. 


Heading information was underutilized. Two possible uses 


were to measure the target trend heading and align 
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the vehicle accordingly, and to improve the general 
sense of spatial orientation of the operator. 

Operators adopted differing strategies for driving. These 
strategies were modified during testing but did not 
converge to a single "correct" method. 

An effective search technique when no target was in view 
was Critical in reducing course run time. 

"Crabbing" or twisting one direction while moving side to 
side in the opposite direction was effective in 
realigning with a new course heading trend while 
keeping targets in view. 

Applying excessive control actions and failing to account for 
the delay caused by the update rate was a problem, 
especially when in close proximity to a target. 

Operators were hesitant to identify target numbers when 
only a portion of the number was visible or the angle 
of view approached 90°. 

As testing progressed, the operators became more confident 
in controlling the vehicle in two or three directions 
simultaneously. 

Operators occasionally applied the correct axis-control order 
in the opposite direction from that intended. This 
was related to the well-known outside-in versus 
inside-out viewing problem. 

Operator average velocities increased as familiarity with the 


testing improved. 


59 








с I TT 3 ГЭ 24  ұұ lj $22 27V] S S SS" mmm" "—" X553 





Chapter 7 


Summary, Conclusions and Recommendations 


7.1 Summary 


Effective collection, computation, and presentation of undersea data to the ROV 
pilot on a single screen presents hardware, software and human-factors problems. Pilot 
performance efficiency through improved display design is a key human factors issue. 
Seven tests measured the effects of an enhanced 3-D perspective display on the simulated 
piloting of an ROV. The effects of the ROV simulation and operator learning curves were 


removed to directly compare performance changes due to the various enhancements. 


7.2 Conclusions 


Display enhancements that improved operator performance were: 
e widening the field of view. 

e providing a gridded screen reference. 

e using vertical color schemes to cue depth/altitude information. 


e shortening the display update period. 


The singularly most important factor for the operator was widening the field of 


view. Providing the operator with continuous range information to the target center 
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degraded performance. 


Many operator comments carried strong implications for system design. 

e The "cross-hairs" effect of a grid aided steering effectiveness. 

e Hidden-line removal for grids could provide strong visual-range cues in the 
immediate vicinity of a target by showing the target "poking through" the 
grid. 


e Color provided spatial visualization cues and altitude slope information. 


Range information proved valuable if a target was not visible, was close, or had 
been missed. 

e The update rate was more important as the range decreased. 

e The joystick was very tiring to use. Fine z-axis movement was difficult due to 


the stiff pure-force control. 


e Presentation of all information on one screen could ease operator workload. 


Several operator comments were not supported by the test results. 

e The 3-D grid was not more effective than the 2-D grid. They were nearly 
identical in overall effectiveness. 

e The color schemes were not far superior to the grids. The difference between the 
2-D grid, the 3-D grid, and the 21-color scheme was insignificant. The 6 
color scheme was only slightly better. 


e The 21-color set was not more effective than the 6-color set. 


Several significant improvements to the proposed 3-D perspective display are 
available through incorporation of the best features of the display enhancements tested. A 
single screen with as wide a field of view as possible and limited distortion, a cross-hair, a 


simple vertical color scheme, target range information, and a reasonable update period (two 
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seconds or less) would improve ROV pilot effectiveness. More testing would be necessary 
to determine the best design for a 2-D or 3-D grid. The key to effective use of range 


information 1s the method of employment. 


7.3 Recommendations 


This thesis investigated the effects of individual display enhancements on operator 
performance. In order to take maximum advantage of the positive aspects that each of the 
enhancement has to offer, several need to be implemented simultaneously. Due to the 
potentially complex nature of the enhancement interactions, it is unwise to generalize the 
individual results into group results. These interactions need to be analyzed on a case 
basis. 

The presentation of both printed information such as position and velocities as well 
as graphics on the same screen was recommended by all operators. This capability exists 
in modern computer windowing environments. The effectiveness of any proposed screen 
layout design requires both qualitative and quantitative assessment. 

One operator commented that a sliding color scale with the vehicle always at scale 
center could provide improved relative-altitude cues. This suggestion has merit and bears 
further investigation. 

Both operators and pilots expressed strong dissatisfaction with the Measurement 
Systems, Inc., joystick designs (fourth-axis control by lifting/depressing the stick vertically 
or pure-force button). The sticks are far too difficult to use for extended periods due to 
their bulk, stiffness, and awkward incorporation of fourth-axis control. Catalog search has 
yet to produce a fully acceptable alternative, although a different joystick model is currently 
in use. Pilot preferences indicate that a modified video-game joystick is closer to their 


concept of an acceptable control. Identification of design requirements, development, 
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testing to ensure effectiveness, and procurement of a fully acceptable control is an area 
needing attention. 

Between testing sessions, usually a day or two apart, operators experienced a decay 
of previously acquired skills. The driving of course X prior to each test appeared to 
reverse this decay. ROV piloting is not a full-time occupation. Frequently months pass 
between cruises. Pilots also change employers, reducing the corporate piloting knowledge 
level. Three actions are recommended to reduce this loss of skills: 

e Holding precruise pilot discussions to reawaken piloting skills. Critical to these 

discussions would be the minutes of the previous post-cruise debriefs. 

e Conduct of precruise pilot simulator training. Construction of a simulator with 
the field equipment would be straightforward and all the necessary hardware 
is already in hand. Pilot discussions of strategies could be demonstrated 
and tested prior to use in the field. Added benefits of a simulator include 
serving as a test bed for new display designs and the ability to write the 
simulator software to incorporate precruise system checks. 

e Holding postcruise debriefs to document problems, fixes, and strategies. The 
minutes of these debriefs would generate a valuable storehouse of piloting 


knowledge. 
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Symbol 


APPENDIX A 


Nomenclature 


Meaning 


coefficient of drag 

look direction matrix from point Р; to 
point Pi+1 in vehicle system 

transformation matrix from vehicle 
system to world system 

vehicle mass 

vehicle added mass 

rotation matrix for K angular rotation 

rotation matrix for ф angular rotation 

rotation matrix for œ angular rotation 

j-th data point 

maximum thrust 

operator field of view angular width 

vehicle system rotation angle about z axis 

world system rotation angle about z axis 

standard deviation 

linear drag (damping) coefficient 

world system rotation angle about y axis 


world system rotation angle about x axis 
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Units 


kg/m or kg-m 


none 


none 
kg 
kg 
none 
none 
none 


none 


rad 
rad 
rad 


none 


rad 


rad 


Page 


38 


37 


51 
38 


38 
37 
ӨТ 
37 
35 
38 
33 
26 
36 
51 
38 
36 
36 





ЕТТІ 


s 4 821717 2 
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kg 


rad 


kilograms 
meters 
newtons 
radians 


seconds 


65 





Symbol 


ACM 
BBN 
CCW 
CDR 
CIE 
CRT 


DSL 
DV 

EV 
ЕЈСС 
FOV 
GDM 
HOOPS 
HSB 
JEEE 
IFIP 
ISCC-NBS 
IV 
LCDR 


APPENDIX B 


Abbreviations 


Meaning 


Association for Computing Machinery 

Bolt, Beranek, and Newman 

Counterclockwise 

Commander 

Commission Internationale de l'Eclaira ge 

Cathode Ray Tube 

Clockwise 

Deep Submergence Laboratory 

Dependent Variable 

Extracted Value 

Fall Joint Computer Congress 

Field of View 

Graphic Display Monitor 

Hierarchical Object Oriented Picture System 

Hue, Saturation, and Brightness 

Institute of Electrical and Electronics Engineers 
International Federation for Information Processing, 
Inter-Society Color Council - National Bureau of Standards 
Independent Variable 


Lieutenant Commander 
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Page 


les 
23 
38 


19 
13 
38 


22 
10 
134 


2] 
25 
10 
133 
134 
19 
22 
136 
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LCNMT 
MIPS 


MFLOPS 
N/A 


NTSC 
RGB 
ROV 

SE 
SHARPS 
SONAR 
SSD 


UCL 
USN 
VCR 
VGA 
WHOI 
YIQ 
2-D 
3-D 


Learning-Corrected, Normalized Mean Times 
Millions of Instructions Per Second 
Massachusetts Institute of Technology 
Millions of Floating-Point Operations Per Second 
Not Applicable 

Nippon Electric Company 

National Television System Committee 

Red, Green, Blue 

Remotely Operated Vehicle 

Standardization Factor 

Sonic High Accuracy Ranging and Positioning System 
Sound Navigation and Ranging 

Summed Standard Deviation 

Texas Instruments 

Universal Color Language 

United States Navy 

Video Cassette Recorder 

Video Graphics Adapter 

Woods Hole Oceanographic Institution 
Luminance, In-phase, and Quadrature 
Two-Dimensional 


Three-Dimensional 
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APPENDIX C 


Target and Grid Design 


All seven tests used the same 25 cubic-wireframe targets. Each target was one 
meter on a side with the invisible x, y, and z data point at its center. Target faces were 
parallel to the planes formed by the world coordinate system axes. Target colors were blue 
except for tests 5 and 6 as discussed in section 5.7. Each target was numbered with white 
numerals (1-25) corresponding to the sequential number of the data point at its center. The 
numbers were 2-D wireframes 0.5 meters in height, 0.3 meters in width (numeral one was 
0.1 meters in width), and drawn parallel to the XZ world coordinate plane facing the initial 
position of the vehicle. All numbers were drawn centered in the target in the x, y, and z 
directions. 

Embedded in each target was an invisible sphere of one-meter diameter. This 
sphere defined the "hit" criterion for its target. When the vehicle, modeled as a point, 
touched the sphere, the target was hit. This corresponded to closing the range to the target 
center to 0.5 meters or less from any direction. 

The 2-D grid used in test 2 was composed of 16 black squares arranged in four 
rows of four. The grid was drawn centered, parallel to, and aligned with the edges of the 
display face at a screen depth of five meters. Each square was scaled so it was one meter 
on a side at that depth. Figure C.1 shows the operator's display of the 2-D grid with target 
course X. 

The 3-D grid used in test 3 was composed of 200 black squares and rectangles 
arranged in three panels. The center panel was made up of 100 squares in 10 rows of 10. 
This grid was drawn centered, parallel to, and aligned with the edges of the display face at 


a screen depth of five meters. Each square was scaled so that it was 0.25 meter on a side at 
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that depth. The side panels were each made up of 50 rectangles in 10 rows of five. Each 
rectangle was 0.25 meter in height and one meter in width. The side panels were attached 
to the center grid at nght angles to its left and right edges. The assembled grid formed a 
box with no top or bottom facing the operator. Figure C.2 shows the operator's display of 


the 3-D grid with target course X. 
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Figure С.1 2-р grid (operator's display with 60° FOV and target course X) 
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Figure C.2 3-р grid (operator's display with 60° FOV and target course X) 
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APPENDIX D 


Data Point Courses 


Tables D.1 through D.19 list the x, y, and z positions of the centers of targets 1 


through 25 for each of the target courses. The method for generating these x, y, and z 


positions is discussed in detail in section 5.2. Additionally, the shortest time difference 


(best dt ) between the centers of successive targets is listed for courses A through R. The 


best Ot between targets U and V was calculated as follows: 


altitude distance from zy to zy was calculated, then divided by the best up/down 


speed (0.25 m/s), resulting in the best up/down time. 
the change in vehicle heading at target U to fly directly to target V was noted. 
distance in the horizontal plane from ( x, , yy ) to ( xy , yy ) was calculated. 
this horizontal distance was decomposed into forward and lateral components 
based on the vehicle's heading prior to reaching target U. 
the decomposed forward distance was divided by the best forward speed (0.5 
m/s). 
the decomposed lateral distance was divided by the best lateral speed (0.25 m/s). 
the minimum time to travel the horizontal distance without heading change was 
the larger of the best forward and lateral speed times. 
vehicle heading change was divided by the best rotational velocity (0.157 rad/s). 
the total horizontal distance was divided by the best forward speed (0.5 m/s). 
the minimum time to travel the horizontal distance with heading change was the 
sum of the best rotational velocity time and the best forward speed time. 
the minimum time to travel the horizontal distance was the smaller of the times 


with and without heading change. 


a 





e the best time between targets was the larger of the best up/down and horizontal 
times. 
Figures D.1 through D.18 were reproduced from photographs of the operator's 
initial display for courses A through R with a 60° field of view. Figures D.19, D.20, and 
D.21 were reproductions of the initial display for course X with a 60°, 30°, and 15° field of 


view respectively. 
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Table D.1 


Course A 








Target 









a 
| 
75.683 


75.340 


75340 | 
75.683 | 
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10 
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Table D.2 Course B 


[ox | у | z | ако 


80.191 | 


16 
17 
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Course A (operator 


Figure D.1 





with 60° FOV) 


display 


Course B (operator's 


Figure D.2 
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Table D.3 


Course C 





pox | у | 2 | ако 





| 6 | 75592 | 20763 | 48232 | 9309 | 
[8 | 77.675 | 39396 | 46.586 | 9855 | 
| | 78482 | 44192 | 45.423 | 9690 | 

13 
14 
15 












E 





17 
18 
19 
20 
21 
22 
29 
24 
25 
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Table D.4 


Course D 





















[oy | » | Вы & _ 
[9917 | 50.658 | 9. 










9917 | 50.658 


74.372 


74372 | 
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| 2000 
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Figure D.4 
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Table D.5 


Course E 





[ox | у | z | Best & 
N/A 

9.752 
9.649 
9.301 
9.378 






| 6 | 68214 | 2806 | 55718 | 9.917 
9.357 
9.209 
9.599 
9.663 
9.905 

23 
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Table D.6 


Course F 









__у | » | Ве & 
| 9822 | 50957 | 9651 





74.090 21522 50.957 9.651 


72.789 
| s 11 7138 | 23500 | 56.109 | 9.130 | 






74.090 | 
12.789 | 





Шш 

EE C 

| 8 | 69282 | 31669 | 59737 
| 9 | 6143 | 42322 | 60019 | 992 | 
do | 65537 | 46929 | 60.170 | 9991 | 
Ги | во | 51277 | 6031 | 997 | 
| 12 | 6100 | 55811 | 59923 | 9936 
| 13 58565 | 60063 | 58930 | 9756 | 
[м | 56794 | 64739 | 58944 | 9864 | 





| 16 | 5135 | 73023 | 57847 | 9.966 _ 
__17 __|_ 49014 | 77376 | 57080 | 9867 | 
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Figure D.5 
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Figure D.6 Course F (operator's display with 60° FOV) 
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Table D.7 


Course G 










5 
WEA 
| 3 __|__ 80930 | 12001 
13.316 
5 13.069 


6 | 95045 | 13750 


12.432 
| 8 | 104761 | 12926 
| 9 [| 109181 | 12034 
10.074 
7.125 
7.113 
8.030 
10.445 
14.175 
16.656 
18.362 
19.258 
18.207 
15.938 

21 14.134 
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Table D.8 


Course H 








| z  [ Best бі 

5 N/A 

9.488 9.010 

14.085 9.987 

18.154 10.684 
5 . 





| 5 | 68558 | 21353 14.338 
Ш. a 11.093 
25.325 9.566 
| 8 | 55.907 | 26.833 9.558 
| 9 | 52.630 | 30.596 9.227 
35.556 9.426 
40.348 9.281 
44.961 9.166 
48.593 9.257 
52.163 8.822 
55.081 9.230 
57.474 10.907 
57.996 14.981 
58.438 19.094 
57.500 17.712 
57.006 19.799 
16.818 
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Figure D.8 
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Table D.9 Course I 









[= | ве 
| 73705 | 9827 | 49845 | 9. 
13.348 


нэг. 

EE 01 
49.845 

ШЕРТЕ 


5 
3 




















1 

2 

3 

4 

5 

7 
10 
11 
12 
13 
14 
16 
17 
18 
21 
22 
23 
24 
25 


ИНГЭ 

S M. 

Ra 

la 

RZ 

0 

L 1 

NE 

[9 | 58587 | 38275 | 45893 | 971 | 
шей ы | 
и | з | 

| | | 5 | 
[13 | 47180 | 43768 | 58279 | 14.261 
м | 
Гм = 

и 45 

явган | 


| 21 | 35754 | 18089 | 81262 | 9452 
| 22 І 31130 | 17307 | 79.527 | 8462 
__23 | 26277 | 16223 | 80051 | 9048 | 
| 24 | 21766 | 15832 | 77931 | 8.483 
| 25 | 17182 | 17512 | 76854 | 9276 | 
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Table D.10 


Course J 









| Target | x | y [| z | | Best & | 
| 6 | 72796 | 29.809 
34.662 
| 8 | 7015 | 39447 
| 9 | 69008 | 44314 
49.184 
53.888 
58.330 
62.885 
67.411 
72.019 
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Table D.11 


Course K 














pox БЕГЕН БЕГЕШ ЕСТЕЛЕ 


га 

E 

Lg 

ГҮ" 

222 | 

aM ЗЭ 2 

CR A 

ШЕ: 

__о | в. 

| 10 | 66.055 | 48.008 | 57501 | 250 _ 
[и | 64013 | 52139 | 50440 9186 | 
I | 56088 | Тал 373 Faces 196 
| 14 | 58333 | 64362 | 65.889 | 9.812 
| 15 57201 | 68246 | 68827 | 11155 | 
| 16 | 56308 | 72.084 | 71905 | 12311 _ 
__17 | 55328 | 75.833 | 75.064 | 12.636 _ 
| 18 | 53753 | 79760 | 77.729 | 10.659 _ 
| ЫШ 
er 
| Bu 
а 
| R 
E 


| 25 | 44374 | 101.705 | 102.813 | 16.618 _ 





1 

2 

3 

4 

5 

7 
10 
11 
12 
15 
14 
15 
16 
17 
19 
19 
20 
21 
22 
25 
24 
25 
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Table D.12 Course L 


Г [шт | Bà 
9.983 | 50415 | 


9.983 | 50.415 
75.020 


74.980 


74980 | 
[ 75000 | 
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Course K (operator's display with 60? FOV) 





Figure D.11 
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Figure D.12 Course L (operator's display with 60° FOV) 





Table D.13 


Course M 





| Target | х | у | в | Best &_| 
[6 | 73107 | 28957 | 55.080 | 9.601 _ 
[7 | 70368 | заш | 55573 | 967. 
8 | emo | 37123 | 56946 | 9615 | 










1 
| 9 | 64808 | 41.067 | 57934 | 9787 | 












91 





Table D.14 


Course N 





| Target | x | у | z | веќ | 
| 6 | 81786 | 28060 | 55718 | 9917 _ 
| 8 | 85209 | 36764 | 59107 920 | 
|. | 86375 | 41499 | 60302 ^ 9599 | 
| 
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Figure D.13 Course M (operator's display with 60° FOV) 
Figure D.14 Course N (operator's display with 60° FOV) 
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Table D.15 Course O 


| Target | x | y | z | веке | 
9.105 
9.130 
9.453 
7 9.185 
80.718 9.897 
| 9 | 82527 | 42.322 | 60.019 | 995 
17 


| 
| 
| 
| 
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Table D.16 


Course P 












Eos ll sc 
| І o | 


| 6 | 54055 | 13750 | 44040 | 9983 | 







|. 8 | 45239 | 12926 | 42.829 | 9532 | 
| 9 | 40819 | 12034 | 44988 | 8.637 













25 








display with 60% FOV) 


S 


Figure D.15 Course O (operator 


| 
«З 





Figure D.16 Course P (operator's display with 60% FOV) 
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Table D.17 


Course Q 








| Target | х | y | z | Вам 
0 NA 
[6 | 84785 | 23831 | 60286 | 11.093 
|. 8 | 9409 | 26833 | 02.137 | 9.558 | 
— {nao С | sm 
ЖЕТЕН ЕТТЕН be | ams | sza З 

12 
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Table D.18 


Course R 





| 9.654 _ 
9 





ы ай 
5 
| 76295 | | 49845 _ 






9.654 


76.295 9.827 | 49.845 | 
13.348 
17.733 





5 20.735 
24.027 


28.690 
| 8 | 90273 | 33.487 
| 9 [| 91413 | 38275 
41.610 
44.963 
40.286 
37.783 
34.940 


32.056 
28.706 











24.785 
20.716 
18.089 
17.307 
16.223 
15.832 
25 17.512 
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Figure D.18 Course R (operator's display with 60% FOV) 


29 





Table D.19 


Course X 








| Target | хо | у | 200 


6 | 78169 | 29761 | 5009 
| 8 | вози | 39441 | 50.037 | 
[9 | 81479 | 44352 | soom - 
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Figure D.20 Course X (operator's display with 30° FOV) 
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Figure D.21 Course X (operator's display with 15° FOV) 
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APPENDIX E 


Raw Data 


Tables E.1 through E.12 list the data for each of the four operators over all seven 
tests. The tabulated values are the time differences between successive targets in seconds. 
Value 1 is the time difference between hitting targets 1 and 2, value 2 for targets 2 and 3, 
etc. The total for each column is indicated at the bottom of the column. For all tests the 
course letter is specified at the top of its respective column. For Test 1 the operator's field 


of view 1s also shown. 


103 





Table E.1 Operator I, Test 1 


30° FOV 

ау ів Ге в [Ге | нј сј еј 0. 
НЫ is | 16 | з | 55 | 53 | 35 | 12 | 531 42 | 
| 2 | 10| 17 | 39 | so | so | 38 | 39 | зо | 49. 
| з | 141 15 | 16 | зз | 67 | ої | 28 | та | 29 
4 | 15 | зт | 2 | wo | 78 | 59 | зз | зі | 45; 
| 5 | 15 | 22 | 32 | 36 | ro | 48 | 35 | 221 79 | 
КИ | 15 | 22 | 22 | зз | 55 | за | 75 | 50 | 43 
НИ | | 5| «| з | 27 | 23 | r | 96 | 32 | 
ps | п | m | 28 | 194 | 25 | з9 | 83 | 60 | 33 | 
[9 | п ај ој as | п| u | 76 | a | zi | 
Рю | п | 35 | зв | 64 | 32 | 3 | 92 | зз | 27| 
и | 18 | 29 | 180 | 31 | 82 | 22 | so | 43 | 189 | 
_12 | 171 241 67 | 116] з | «| з | зі | 2a | 
[13 | 15 | 18 | 34 | 75 | 1 | 106 | 63 | 19 | 46 | 
[14 | 1| 20| 18 | з9 | 68 | «a | ој ој а! 
[15 | 46 | 19 | 25 | 73 | 69 | 94 | 32 | 26 | 59 | 
[16 | 36 | 49 | «| т | 157 | 52 | аз ^ 03 | 109 | 
[19 | 13 | 18 | 104 | S1 | 21 | 46 | 89 | 25 | a | 
[20 | i2] i | 38 | 85 | a | во | 35| 21| яз. 
[2 | nu | 16] 38 | ag | п| 2 | 41 19 | 2 | 
[23 | 10 | 18 | 27 | 152 | m | 42 | 33 | 16 | 28 
[24 | 66 | 12 | m | 52 | 18 | бо | 32 | 13 | 188 | 
Toat | 432 | 535 | 1218 | 1488 | 1120 | 1317 | 1253 | 1036 | 1741 | 


МО I 

O [оо 
Т4| 5 15512212: 

Со |4 M 
I I [о IS Je 
I Iw (ІО | ба 
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Table E.2 Operator I - Tests 2, 3, and 4 


MEA LEN E Y ON [o la je lr 
__ | 16| 161] a | 15 | 3 | 20 | 18 | 19 | 16 
ЕД | "“Г ој мІ ү | 13] з] 2 | 
ВА. it | 15| 211 лој 15| | nu | 16 | 18 
са [І 17 | 29 | 28 | 14 | 22| 28 | 14 | 23 | 28. 
[5 | 24| 46 | 19 | 18 | 18 | 36 | 15 | 13 | 36 
ВЕ! 09] в 55 | 91] 3| 1| 5 
_7 | 2 | т | м | зв | 22 | 27 | 16 | 25 | 19. 
EN .2| s] sp | з | з | uj 12] 47 
L9 | «| 2 | з | 14 | 15 | 17 | 10 | 20 | 16 | 
БИК | > | a| зг| 15| oj 5 | uj w| mi 
EN | 25 | 15 | 105 | із | 1з | 16 | 5| 131 53. 
се јијиј еј иј вј вј вј цан. 
ЮМ і з] зі із | 12 | «0 | 2 | 2 | 19: 


Нь. 26 | 52 | 47 | 16 | 2 | 12 | uj sj 
|16 | 25 | 38 | a | 16 | 12 | 48 | 12 | 15 | 49 
lm | зв | 34 | 4| 22| 1 | зо] 13 | 13 | 28 | 
[18 | 27 | 17 | во | 16 | 18 | во | 13 | 12 | 20 | 
[19 | 22 | 26 | 102 | 18 | 14 | 70 | 14 | 14] 2 
[20 | 4| 20 | 3 | 25 | 15 | зв | п | 15 | 30 
[21 | і з] 19 | 20] п | 37 ^ 14 | 10| 17 
EE ST »| 2| »]| %®| 16 | 2 | 10 
|a| 12 | 25 | 22 | | 2| 12 | ж] м 

24 
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Table E.3 Operator I - Tests 5, 6, and 7 


17 12 

14 24 24 

18 
pa | 23 | 15| зо| 12 | 33 | 25| 12) 271 зм 
_18 | 22 | 26 | 86 | 23 | a | 32] з | 18 | 60 
je | wf wt 6 | иј мј пј вј yu 
[20 | a | 22 | 28 | u | 14 | aa | 16 | 24 | 25 
E 2| 2) :]| nj 18 | е]! 15 | зз | 2 
2 | 55| 14 | го | 10 | 23 | 17 | 16 | 15 | 42. 


888 | 327 | 430 | 726 | 462 | 619 | 793 


MN ia 
ЕЕ 
= | 
ло 
pla 
Y JN 
Ев 
оо ID 


m [m = 
© IN © Сл IP Io IN 
LJ — | 
NO © --4 
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ОО 
Е 
со 
је 
= 
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Table E.4 Operator II - Test 1 


ЗЕРЕ КЕКИКИЕИКИКИ 
ER] 15 | 32 | зв | 15| з | 45 | 22 | «а | в 
a a з sl al as 
ШИ e | s) 12 | 25 | зо | жа | 33 | 21 4 
| 6 | wo | 35 | 97 | 15 | 53 | 52 | 67 | 14] 19 
__7 | зе | 23 | 20 | 76 | 29 | 35 | т | 2 | 47 
[8 | вё] мі 75 | 15 | 30 | 52 | 78 | 92 | 88 . 
[9 | 19 | 15 | 239 | a | 36 | т | 32 | 37 | аз. 
BĘ | s | r | m | 16 | 33 | 23 | 32 | us| m | 
и | 9 | a | s | uć | e | 16 | 34 | 48 | 87 | 
[2 | m | 44 | во | з | 24 | 93] 36 | зо І 115 
213 | зі | 20 | 55 | зо | 26 | 185 | 25 | s9 | ов 
lu | 6 | зо | 54 | 59 | 7 | 27 | «| 9 | 94 | 
pis | 19| 21| 56 | зо | 13 | 4 | 36 | «| 86 
[16 | 25 | 27 | 38 | по | 49 | з | 2 | 9 | 47 
17 1 14| 20 | 259 | 87 | 16 | 63 | во | 245 | зе 
[18 | 82 | 20 12 | зі | 2 | 5 | 17 | 56 | mo | 
[19 | 42 | 16 | 87 | 6 | 24 | а | 34 | 130 | го 
[20 | 27 | 13 | 66 | 44 | 35 | 68 | за | 26 | 13 
la | nu | 12 | 85 | бој 43 | sr | 24| 43 | 113 
[22 | 19 | эт] за] 52 | 6 | зо | зі | sz | 97 
[23 | 29 | 1 | “| 421 17 | 29 
la | s | ој мј mo | 86 | 19 | 125 | 72 | ві 
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Table E.5 Operator II - Tests 2, 3, and 4 


Test 4 

[a | p | о | в | |н |; 

a | o | 16 | a | 12 | 25| 3 
НЕЇ | o| з | | so | v. 
а | 13 | 55 ^ 26 | м | 36 | 30 
ЊЕ | 5| за | г| с| гј 43 | 
ЊЕ | 20 | 22 | 25 | 22 | 16 | 82 
МЕ | u | 25 | 17| 221 22 | 5 
ER | з] п a | s | 59 | о! 17! 
L9 | 161 25 | 23 | 32 з | 1 | ој 17: 
do | 19 | 33 | 45 | 33 | 13 | 23 | 10 | 19 | з 
lu | 25 | 32 | 76 | 33 | в01 221 151 1217 
s | 2 | 2 | 45 | зо | a | 27 | u | 17 | 79 
16 | 27 | 28 | зв | з | 48 | m | 461 30 
| 78 | 36 | 30 | m | m | 16| 3 
" "UNE TEN т вт ит. 
| 25 | 99 | 25 | wo | 66 | 18 | 12| 16. 
По а | Гир а 
a_a Em [s [oss [s | n 
[22 | 18 | 22 | | зо | 49 | 65 | 25 | 14 | 20 
e | | 6| «| m) 26 5| 2. 
_2 | 35 | m | зә | 28 | 36 | as | m | 1s | 48 
Total | 433 | 700 |1330 | 712 | 731 | 960 | 363 | 414 | 825 


үнэд 
Ww [02 


— m dm 

= u [оО 
w |= ре» ре» te FR |92 |959 
— Iù |- ооо 


W 
~ 


m [LA 

oo |- 
а | | 
o la lu la 

ШІ 

Фо. 


108 





Table E.6 Operator II - Tests 5, 6, and 7 


7 


No ID 
oO IP ISO 
O Io Ime [оо 


EM 

ЫЫ аі Ри ГГ» 58 
Me) a) | п | «| пі эр 
ps | ut at it if із | w | 45 | 28 | a | 
МЕ! “| 15| 37 | 25 | 19] 221 33 | 2 | z | 
_12 | 2 | 43 | 32 | 19 | 22 | 136 | 32 | 23 | м. 
МЕ | 25 | 25 | зо | 16 | 12 | т | 2 | 251 27. 
[14 | 26 | 18 ^ з | 3 | 18 | 87] 14] 23 | 28 | 
НЕО | 2 | 15| 36] 2 | | 2 | “| 261 53. 
218 | w | 28 | sa | 16 | ma | 38 | m | 321 5 | 
ВЕ | | ss | m | s | 12 | 2 | з | 2 | 46 | 
Em | a | 19 | 28 | 15 | 19 | 28 | 2 | 1 | a 
la | 30 | м| 49! т | 211 45 | 6 | nm | 37. 
22 | в0 | u | 42| 16| п | ој 16| 20| a | 
_ 23 | зп | 12 | | 19 | 27 | 42 
Total | 536 | 495 | 829 | 466 | 431 | 1052 | 652 | 609 | 1006 | 
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Table E.7 Operator III - Test 1 


ЕЕ APA E [Pa ote 
| 2 | 121 78 | 16| 1| 40| 49| 1j 0 | зв. 
ЕЗІ ПІ | 4 | 1з | s3 | 6s | w | us | 28 
“| wo | wo | 39 | 64 | as | 88 | 55 | 153 | 287. 
[5 | n] 4| | 2 | a | 471 21 33 | 46 
| 6 | u | м | 18 | в | a | 3% | is | зо | 29 
КО | 10 | 28 | 23 | за | ss] 13 | 29 | > 
28 | 10 | 54| 2 | 6 | «| о | з | 191 3 
| 9 | 10 | 1a | 18 | so | a | 5 | 9| 26 | 124. 
do | з | u | 27 | ој ој ој џ|ј зі | 182; 
u | ој п | пј | 45 | 20 | 14 | пе | 68 
_13 | 14| по | 19 | 76 | 86 | 6 | з | 24 | 101 
41161 31 3a | 68 | 99 | 90 | a | 20 | ті 
_15 | 68 | 17 | 20 | 48 | 47 | 93 | 28 | u | 42 
ВЕ а ы! 25 | 55 | 59Ї зо | 17 Р 271 3. 
de | 15 | 42 | 86 | 65 | в | о | п | 158 | 102 
_19 | еј m | 256 | 70 | 40 | 17 | 10 | 32 | 230 


ги 34 | 

DE] 9/5) | др юр 2 | 17 
ВО | о | 22 | u | ss | 20 | 1 | 13| 621 26 
[23 | 99| 20! 46| 6 | | 26 | ыш ој 23 
[24 | 66 | 38 | m | з | a | 65 | 13 | 10| 4. 
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Table E.8 Operator III - Tests 2, 3, and 4 


E га те Га 

5 | 20 | w | w | 12 | із | 15 | m | 38 | 26 
[6 | 2| a | 23 | aa | «o | u | ss | а | 2 
ШИГ | „| 72 | 2 | 17 | и| el 1) ғ 
ВЕ! | si] at it 20 | ж] ж] | п| 
L9 | з | 22 | n| a | r | м | 16 | за] 73 | 
з | в | м | 65 | з8 | 34 | 26 | 25| 1 | 5 
_16 | 67 | зі | 47 | п | 4 | «| 26 | 16 | 5. 
Po | 27 | 26 | зв | 12 | 22 | 4з | ла | 23 | 37 
[ле | 23 | 21 | a | зі се | п | 19 | 22 | 74. 
219 | 24 | 25 | зо | 15 | зо | ma | за | 16 | 54; 
[23 | | 18 | о | 9 | п | 15] мо Р п | м. 
pa) 6| 12] 21| ој 9 | 55 | з | з] 46 | 


m | M 
M fa 
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Table E.9 Operator III - Tests 5, 6, and 7 


КООШ ге тате 
pi | wf a] 15| п | 26| п | 16] 21 20 
юю юам юзе 

101 20| 23 | 5121 91 21| s | so | 

БЕ | за | зз | 16 | зт | 231 201 231 3! 
а | = | 2 | о | = | 1 | | а 
| 6 | 46 | 14] 202 111 241 451 21 | 20 | 
[8 | 44 P 17 | 30} wo | 17 | зв | 15 | 41 3. 
_> | аі аі 16| п | 13 | 4| 24| 25 | 25 | 
[10 | s4 | 14 | 36 | 10 | 69 | ra | 26 | 191 32 | 
_12 | 48 | 21 | 17 | io | 35 | m | 28 | зо | 98 | 
[13 | 221| 22] 2] 9] mu | в | 56 | 19 | 66 | 
МЕ! 20 | 2 | «| 2 | зо | 75 | ж] ів | 27 | 
_15 | 17 | 23 | 21 | 10 | 26 | в | 19 | 17 | 29 | 
_16 | 15 | 24 | зз | 18 | m | so | зо | 19 | se 
lu | 2 | u | as | 1u | 25 | зв | 17 | 104 | 35 
[22 | 10 | 10 | 52 | 16 | 23 | 65 | 26 | 36 | 171 
[23 | 25 | wf] 2 Р 9 | 19 | 26 | so | 28 | 191 
_ 24 | 23 | 10 | 18 | 16 | 17 | 14 | 24 | 28 ^ 26. 
тош | 542 | 489 | 680 | 370 | 696 | 1082 | 587 | 750 | 1034 | 


TP 
© 
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Table E.10 Operator IV - Test 1 


Кажы Ге Рі Іо |Цж г | м е 
ШО 38 | 23 | 25 | «| з | 79 | 76 | 46 | 35 | 
| 2 | з | 46 | 32 | 44 | 46 | п | в | зо | зі | 
з | зо | 7| a | 38 | 6 | зі | вт | 132 | ав | 
ШО 0 | as] 38 | о | з | ві | 48| a | nio | 
| 5 | sa | m | зв | то | з9 | 83 | 64 | 27 | 54 
| 6 | 39 | «| 2 | зо | 24 | 19 | sa| so | zm | 
(7 | 67 | 25 | 8 | aa | 27 | © | 43 | з | 56. 
[8 | s| 48 | 33 | 79 | 18 | 48 | зо | 2 | 316 | 
229 | 28 | 2 | м | 26 | 15 | iss | u | 51 | mo | 
[12 | 23 | 25 | 20 | 25 | 29 | о | зі | sa | 152 | 
a | 33 | 15 | за | 26 | 37 | 1л | 104 | 202 | 17 
_14 | «2 | зі | зі | m | 42 | 282 | 135 | 216 | 258 
pis | 45 | 19 | 35 | 22 | зв | по | 43 | 45 | 90 | 
_17 | з | м | 46 | 2 | 46 | 95 | 40 | 54 | 96 | 
de | 49 | 18 | ma | 33 | 48 | 6 | 99 | 176 | 88. 
Г э и] з] э simu] see 
_20 | 6 5 | 6 | 12 ej c 
[22 | «| a | о | 19 | 53 | 66| 32 | 40 | 63 | 
[23 | 36| m | 66 | 38 | 35 | що | 57 | « | 121 | 
[24 | 4s | 14 | « | am | 39 | 20 | 60 | 30 | 79 | 
Total | 999 | 734 | 980 | 880 | 945 | 2469 | 1575 | 1907 | 3081 | 
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Operator IV - Tests 2, 3, and 4 


Table E.11 


Ол] sl Ol ol WO] MI CO] NT WOT Ol MI AT] оо мІ соі хо 

Е ыы СЕ! | I I GSO СЧ Ml + O І (С = 

v— | © 
ro 

21 oo| се| хо 

— maj mj = 

С) 

© 

E 


a 


= == NI I =~] N 
са CA] со | оо 
=| =] =| NI N 
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Table E.12 Operator IV - Tests 5, 6, and 7 


тамо [аА {ро[у (м [к 
[4 | 1 | 13 | м| s | во | 2 | 17 | 24] 2 ] 
5 | з | п | а | 1 | зо | 16 | 13 | 23 | 18 
[6| mj| u | 16 | 13 | 12 | 15 | 13 | 22 | 60 
[1 | wo | m | ss | 16 | 15 | 18 | 14 | 24 | 55 
в 1з | u | 16 | s | 14 | | | 25 | 29 
__9 | 12| 12| is} 2] if а5 | 15] 14] 2 
[10 | 17| 17 | зв | 20 | 29 | sa | 12 | 20] 2. 
u | з Д з мІ | so | as | m | 16 | 60. 
_14 | п| | | 12 | 2 | 9 | 25 | w | 12 
Dx s Los posce coco ва, 
20 | 2 | пј 5| и| зј| 221 24 | 1| 31 
[8 | 141 27| о | ој m | ra | 13 | 18 | 2. 
[19 | 12| 141] 45| nu | 26 | 7з | з | 1 | 1- 
[2 | 15| ој з5| 17 | 201 16 | 32 | 23 | 26 
22 | 121 10 | 25 | 13| 171 21 18 | з | 19 
223 | 10| | 14 | wo | 12 | 15 | 14| 2 | 24 
[24 | 9| 10| 25 ^ 48 ! п | as | 26 | 14 | 18 
Tot | 366 | 355 | 817 | 380 | 552 | 900 | 46 914 
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APPENDIX E 


Normalized Data 


Tables F.1 through F.12 list the data for each of the four operators over all seven 
tests. The tabulated values are normalized values from the associated tables in Appendix E. 
The data were normalized to remove effects properly attributed to variations in the level of 
course difficulty related to the Jason vehicle simulation model. The total, mean value, and 
standard deviation (©) for each column are indicated at the bottom of the column. For all 
tests the course letter is specified at the top of its respective column. For Test 1 the 
operator's field of view is also shown. 

The normalization procedure divided each time difference by the best time between 
the associated targets from Appendix D to arrive at the normalized value (dimensionless). 
Because the logged times represented the most recently passed whole second, while the 
best times between targets were calculated to a greater degree of precision, it was possible 


to achieve a normalized value of 0.9. 
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ТаБје Е.1 Operator I - Test 1 


60° FOV 
нев 
1.2 за за] sl] os 
ор зар 21 зв] ваў за 
о з2 291 so] зз] 23 
а о а els] > 
215 | 47{ 21] 15| so] ве] sef 321 26| 36! 
_16 | 38] 40] 241 61| 163] 351 43| 94| 68 
ЕТІН БЕЛ БЕТТЕРІ зала и as] 23] 23 
[o | 13] 201 72| «ol 22] 23] 93| 25 
Mala fast el elas] as] a] e 
а Барај ар ај | sej au] zaj зе 
[22 | гү 16] 42] 30] 11] 26] 36| 191 23 
EX] шү ssl эт] or) гэ] азі 35| 15| 33 
ра вэ] 121 131 201 19, зар 36) 124109 
[wien 1a | заў зар заў asf asf saf asf өз 
Се a] or] 21] 3s] 34] 27] 24| 21] 52 


1 


N 


сии (недај еа ој овј a de 128 
~ lo JR | m [2 a lu la la lO la 


кылын соц а Еш БУ е ea |e Ене 
> мм TN IN | 9. 
E М 
t2 јо IS [л |О је fw fo [A јо јал Ju 
со |.» JO | [с JW fw IS Im TR ре la 
t3 [tA ОО 
v [wv 95, 


а | 
Alu lalo IS (Іо Іса la lo lo Io lu lia 
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Table F.2 Operator I - Tests 2, 3, and 4 


Test 3 

м |--A а 
КО, ыв| is 
EL | 11 13 | 25 
Mi ai 
Га | аз. за] 29 ија 
тир ЫТЫ ы ы ыы 
eela] aed ast soj osf за 151 18 


A 
(ә 
- |2 
L 


ко 
ОЈ 


99 

00 
== = 
ЕН 2 
ND | | 
w [а [oo 
ю Ре ре |м 
о (о [5 Iit 


E nope 
о Го |За 


o 


Га саа гаа за гаа гаа ав заў 20 
ЕГЕ ШЕТІ ны наас 
pk | 4) 22 al as as лар 10 
E 


me Me RR 


2 


кә 

ОО 

tA [t^ [to 

© [W IN 

— |-> 
ia 

E 

UJ 

E 

4 


Ко JO [о 
№ | |е 
= = 
ЕЗ 

— 

рші 

N 

UJ 

= 

es 

= 

o 

= 

=) 


RE | is тей ШЕТІ БЕТІ ЕТІ БЕРІ ШЕТІ БЕТІ 
3. 
a аа зао. 
Сазе аа раза 
ЕГЕ ШЕГІ ШЕТІ ШЕТІ ШЕТІ ШЕТІНЕГІ БЕТЕНЕТІНЕГІ 
22 —25{ 23] 35] 25} 091 321 16) 124 11 
Es [asp asp asp aa pas 12] ај ај и 
24 

1 


ЕГІН ШЕТІ БЕТІ ШЕТІ ШЕТІ 10 0.5 
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Table F.3 Operator I - Tests 5, 6, and 7 


[4 | 15] 14| 26| 09| 16] 27] 181 311 27 

2] ле 18] 12) 10] 20] тј зеј 1» 
Bo |Ma is as as o ај s a 
Du | 42] 18] as] 19] 18] 22] 201 as] 40 
12 | 32| 14] 27| 21] 16| 60| 21| 29| 18 
па зәр ма saf ар лера ла за аз 
[м | 20| 15| 32| 121 351 151 121 30] 37 
pe ала а as Гэ ав zo] a6| ar 
o fas] col м| ај мј 071 11 261 15 
pa |a| | is] u] is] sa] ај за зе 
ЕТЕ ШЕТІ ШЕГІ ШЕТІ ШЕСІ ШЕТІ ШЕТІ ШЕТІ ШЕТІ ав 
16 
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Table F.4 Operator II - Test 1 


60° FOV 30° FOV 15° FOV 


кне о 
BE uu] opu] el ub 
BE |» 5] за 49 22] 46 1.5 
са лаф ав ара 02d 36) аз. 
5 | 121 59| 121 251 32) 183) 34] 40| 42 
Геј ај мр Г sa| за вара ы 
ПИ | 351 24| 22| 27| 31] 32| 72| 29| 50 
[8 | 65| 24| 87| 15| за] 56| 80| 92| 91 
[9 | 19] 17| м3| 25| 37| 79| 33| 37| 46. 
to pza] of за 191 331 asf saf пеј ве 
pe | ae] ar] сэв Ган аа oo| эл аа Га 
[1 | 3i] 22] за 31| 26| дој 25| 60| то 
[14 | 6s| 33| 44| so| 78| 29| 41| 104| 55 
| 5 | 19] 231 зар зар 13] 39] 36] 44| 52 
[16 | 26| 22| 21| 87| si! 21| зо] во] 29 
Eu [asp a] ns] ва ie | за eo zo zs 
[8 | 83| 23| 74| 27| 20| 33| 17| 59| м1 
[19 | 42| 18| 60| 48| 25| 22| 35| 132| 245 
Em | oa гав ы ав мр ај мај ыг 
[21 | тла] 121 961 42| 44| 54| 283| 44| 134 
[22 | 19| 28| 37| 33| 63| 30| 33| 58| 107 
223 | 30| 19| 51| 27| 17| 29| 90| 72] 189. 
[24 | 91| 10| 38| 42| 88| 20| 135] 73| 66 
Тоа | 72.1 | 697 | 1333 | 969 | 99.0 | 126.6 | 133.4 | 196.4 | 211.6 
[с | 23| 18] 36| 27| 21| 47] 55| 64| 61 


, ы 
ON 


120 


11717220 11 " й 


| adm 


m 0] 1! 





ы 


re © 






Table Е.5 Operator II - Tests 2, 3, and 4 


Веро Ге Ee | нј r | N | R 
ШІ 21 2 29] 22] 26] 17] 12] 131 33 
ET L6| 10 
aaa Das zz Dae ај а ај ај а 
ШЕ! 254 35] 21) 16] 23| 39] 14| 16] 18 

sa | in| ose 12] 25] 20 
[3 ЕРТЕ ШЕТІ БЕТІ БЕУІ ЕГІ БЕТІ БЕТІ 25| 12 
= | за саа 201 за] за] «Гер ара 
Ra | 16] 2s] 07| заў 25] 18|] м] 18| 39 
PB | 19] 291 26| 361 131 251 10| 19] 29 
_п | за 330 мај aot 18, 20) 101 161 si 
Го | as] 24) s3| во 16| 18] 13] 13 
13 | 20| 156] 37| 36] 151 ав] 181 10) 21 
Du [aep asp sp zs si] aa] ыра 
[15 | 20| 23| 28| 24| 31] 25] 14] 17| 48 
МЕ | 131 21| 41| 34| 30| 23| 17| 16] 24 
ЕСІН БЕРІ ШЕТ arj 281 soj soj 171 22 
КЕ ваза ааа рае 
[22 | 18| 23| 66| 19| 49| 64| 25| 14| 22 
pa | лар 6] sa] 2o| sz] а 281 091 2. 
A p sa| al aaf arj 321 ы 181 11-32 
Mean | 18 | 30| 41| 27| 31| 35| 15| 18| зо 
[о | 06| 24| 26] 10| 15| 19] 05| 08| 12 
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Table F.6 Operator Il - Tests 5, 6, and 7 


НЕЕ ьм а 
a 3 за зор з]. аа. 
a || au] so] зт] аз] зз. 2] 20] за 
11| зо 2а | аз | за 
БЕ Ыза 25| 26) 21| 111 37| 54| 22| 28 
5 |а| оз | 251 1а| 28| 29| 29| 22| 45 
ПИ ma 22] 2] ры 
RZ | 10] 211 151 17] 14] 141 231 27| 29 
| 8 | 11| 25| 24| 17) 13) га] ав] 29| 23 
Ки | :л| :5| 22] 26| 19| 23| за] 241 29 
и | 20| 14| 27| 201 181 52| 20| 22| 40 
pe | sa] мав] зо] за] о] за] ај 15 
ШЕ. 25. 25| 27| 16| 121 51| 221 27| 31! 
[1 | 22| 19] 35| 23| 19] sa] 14| 2s| зе 
| 5 | 241 13 22] 27| 17| 14| 26| 29| 49 
pa | 16| so] sof ae] 22] es] os] as] 28 
20 | 15] 19] 211 16] пој 30| 24| 21| 24 
ЕТЕП ЕЕТЕ ШЕТІ ШЕГІ ај 21] 53] 211 ав | 35 
[22 | si| 11) 46] 17) 11] 104) 17] 21] 63 
a [os] 21 зә] зар лер зар 201 27 аз 
ЕТІН БЕРЕЛ БЕРІ БЕТІ 
62.3 
[Mem | 19] эл] 28| 20| 19| 39| 28| 26| 33 
o | os| os| m| os| os| 22] 11| os] 16 
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Table F.7 Operator III - Test 1 


MAT MERC? RAT [a ola 
ИГЕ ШЕП ШЕГІШЕТІ БЕТІ ЕТІ ШТІ БЕТІ ЕГТІ БЕТІ 
Га [леў заў 27] 65] 49 102 | 55 | 165 | 299. 
BĘ | u] 36] 19| 711 511 аз] 18| зі] за 
L.8 | 10| 56| 29| 78] 441 175] 13] 19| 411 
ped Га Геј ај ој ај и 
Tal al ul sal sa] ај ај 30 
Ea Me Da za а аа нар зар зар зе 
Lu | aa] sal 3a] 63] m4] sel 25] 22 
ШЕ | | al el elos] | эў 26 
_16 | 45| 12] si] 571 44| зо] 43| 30| 39 
о | мој саа аз] ss] es] 20] БЕТІ БЕТІ БЕТІ 
[18 | 13| 43| 49| 67| 651 Bo| 11| 178] 57 
[19 | 491 17] m9] 73] 41] 19] 10] зе] 159! 
[m | so| зер воў за 46| 18| o9] 37| 292 
a | 13| 31] 43| 751 541 251 101 221 143; 
(2 ШЕТІ НЕГІ ШЕТІ ШЕТІ ШЕТІ ШЕТІ ШЕСІ ШЕТІ БЕТІ 
la [ssl col e] sl sl за аа Гаваа. 

24 
тои | cos | 796 | 924 1465 1289 | 1609 | 4851 1196 

Mean 

НЕ за зо! 25] 16| 49] із] 53' 
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— бэ 
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Table F.8 Operator III - Tests 2, 3, and 4 


Гала а ој ај ве 
ИЕ 31| 12| 18| 10| 19| 22| 37] 12| 37 
КЕ | 34| | 26] 33| 23| 12| 27| 18] 88 
a Pas] ар мр ар ај ај ај ај 26 
S| 20] зе] паа]. 241 151 aL 

pa | 29] 22] 24] 45| 44| 12 
ШО Жү | 75] 12] 17] 15: 
pa | 20] 32| 47| 18] 20] 22 
Ви | 14| 29| 18| 22| 79| 15 
Гао | 24 15| ваў 24| 64| 20 
ПЕ a so] а 22| ы 
БЕТІН БЕТІ ШЕГІ ШЕТІ БЕТІ ШЕТІ ШЕГІ ШЕТІ 14 23 
со | ва tel sel sol 34) 181 251 131 за 
ШЕТЕН 20} sat ot 11) all 101 161 27 
as ле зар asf sol 16[ 29] ај а 
[m | 251 жүз 12] 24| 29 17 23 19 
Св гаа Гал ao] за] es] аа аа аа аз. 
-22-1-281 ы) а ы зо 224 191 їл] 25 
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Table F.9 Operator III - Tests 5, 6, and 7 


Pas as ul as al af as эг 
2 aż afse 
aio asf ш ss] э ы ы ssf es 
[4 | 13| 63| 23| 16| 40| 24| 201 251 40 
КЕ | 22| 15| 21| 12| 72| 15| 12| 35| 47 
[6 | 47| 15| 21| 11| 25| 48| 21| 29| 22 
ВЕ | «| i| 33] 10| 17| 42| 15| 46| 3 
BE) <> | 23! 17) 11] 14] 29] 24| 26| 27 
mo | sa] 12] ao] rof во] ыр 26 
pa НЕТ НЕТ а ае за аа зо зе 
[12 | 50| 23| 18| 10| 38| 391 281 31 
[is | 24] 24| 25| oo| 34| ar] sa| 19] 47 
Du | 201 за as] 12] | ва] 23] 328] 16 
BE | 15| 20] 22] 19| 26) 27| 31| 20) 36 
pu | а] ne] зар ер за заў se] за] за 
39 
-2 paj tod ле за] 71 26 26 19 
а рај лар зава лар вар ар зар 22 
А 25] 10} 19} эл} ы] 16, 251 281 28 
EN O EI 
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Table F.10 Operator IV - Test 1 


60° FOV 

к | мјо | L |o | R | | м | г. 
ma з8| 48| 32| 44| 48] 85| 86| 31| 54 
| 4 | 50| 27| 27| 63| 43| 70| 48| 66| 114 
ШЕ | зэ 52| 21| 40| 26| 21| 53| 52| 296. 
[7 | 68| 27| 24| 44| 27| 65| 43| 76| 59. 
ЩЕ 52| 50| 36| 82| 18| 49| 30| 25| 366] 
НЫ | 29 43| 25|. 27| 15| 195| 21| 56| 84 
ШЕ | 34| 15| 39| 26| 38| 121| 105| 220| 1241 
Ги | 36| 32| 34| 41| 44| 164| 139 | 236| 212 
гав | за зар за] аа аа аа Гэв cs 
Eu a 24} зе. ы ва] чар еј з 
Dak a 221101 рај «о 
о за| 22| 17| 30| 15| 221 128] 65| 72 
EN LL 4| 23] зві з] 25] зз] 121] 74| 47] 
са | 231 421 39 201 ssl 131 sa| aij so 
-z | as) saf sri 401 251 mel so 4.2 
Total | 888 | 75.6 | 83 4] зов] эво [тоз [зәл | 2001 | 2026 
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Table F.11 Operator IV - Tests 2, 3, and 4 


EEC OP 
a | иј 1a] 21| за] 16] 22] 121 tal 30 
pa | mo] 231 131 12] sol 2еј 221 is es 
[3 | 16] 23] за] о | 23| 19| 18| 26| 20 
[4 | 17| 29| 29| 12| 18| 61| 36| 28| 24 
[6 | 14| 22| 22| 15| 17| 24| 16] 19| 156 
pas] as] з рф щі зріз 
BE | 23| 2з| 21| 23| 19| 14| 11| 30| 25 
јо | 21| 20| 20| 19| 18| 23| 12| 18] 27 
[2 | 20| 14| 63| 18| 32| 54| 18| 16| ва 
[is | 14] 10] 23| 14| 16| ss] ла заў за 
pea] пеј 2st | az | бз] ы 181 18 
[5 | 12 2 ааа. «аа 15] 18] 22 
о | зој E за] ај ај ај ар мо 
о | ај о ој змај о. мр ма 
араў Гара 
eos ss psa 
[23 | 14| тај 221 aif as] 15] заў тор 11 
га раа шј ај 10] Г зер so as 
459 | 723 | 3&1 | 459 | бөө | 465 | зл | 730. 
«= os} is] за us 18 [08| 08| 16 
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Table F. 12 Operator IV - Tests 5, 6, and 7 


моја | ово A E 
ШЕГІ БЕТІ БЕТІ ШЕТІ ШЕГІ ze a 

БР сој С| | 
3| 11] 19| 12) за] 16| 131 231 18 
ыы ы ыш ue] us] ај за 
9| 14 
а 
id. is 


1.0 


sl sl ia sf а] asf ar 
БЕТІ ШЕТІ ШЕТЕШЕГІ ШЕТІ ШЕТІШЕТІ 

ЕГІН аа БЕТІ ШЕТІ зо] заре 20 
[4 | a] 24] 21| 12| 24| sıl 26| 201 100 
As ns зэ} tif 261 481 26, 111 21 
ШЕГІ ШЕТІШЕТЕ 
ЕТІН БЕСІ ШЕРІ ШЕТІ ШЕГІ ај зар 1а] 2s 
psf os) ssf if за 22 a 
E» | is] тој 251 13| 17| 24| 18] 13| 21. 
ps | u| У а зер за 
pa | 10] тој за зала вр аа 14 19 
Total 36.5 | 683 | 38.5 
LO 52| 09| 13[ 15[ 07[ 06| то 


= 
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APPENDIX G 


Operator Learning Curves 


Estimation of the operators' learning curves involved four assumptions. 

e The learning curves had positive slope at all times. 

e The learning curves were smooth functions. 

e The learning curves were exponential functions. 

e For a specified field of view, operator learning was reflected primarily in the 


standard deviation of the normalized course times (a smoothness measure). 


Each operator drove 21 courses with a 60° field of view, three for each of the seven 
tests. The procedure below was applied to the normalized data for each operator to estimate 
his 60° FOV learning curve. 

e The standard deviations were entered into a 3 x 7 matrix. Each row represented 

a specific window width B as discussed in section 5.2 (row 1 = 15°, row 2 
= 30°, and row 3 = 60°). Each column corresponded to a given test (column 
| = test 1, column 2 = test 2, etc.). 

e The standard deviations for the three window widths for each test were summed 
and entered into a row vector of seven elements. Each column continued to 
represent a given test as above. This step yielded the summed standard 
deviations (SSD). 

e The row vector elements were plotted with summed standard deviations on the 
vertical axis versus test number on the horizontal axis. 

e An exponential curve using the method of least squares was fitted to the plot by 


linearizing the exponential form through taking logarithms [G-1]. This 
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defined the operator's 60° FOV learning curve. 


Figures G.1 through G.4 show the learning curves (graphed as lines) versus the 


test number. 
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SSD 4 





Test -> 1 2 3 4 Э б 7 
Figure G.1 Operator I 60° FOV learning curve 
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Test-> 1 2 B 4 S 6 7 
Figure G.2 Operator II 60? FOV learning curve 
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Test -> 1 2 3 4 5 6 7 
Figure G.3 Operator III 60? FOV learning curve 
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Test-> 1 2 3 4 5 6 Я 
Figure G.4 Operator IV 60° FOV learning curve 
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